We each have a magic genie that will do whatever we ask of it. So do we still need anything from other people? By the same logic, if everyone else has a magic genie that does whatever they want, why would anyone need something we make?

Another way of saying that is: if the magic genie makes creation abundant, what will become scarce and valuable?

Of course the magic genie is AI. The question of the hour is: if AI serves as a magical personal assistant, or a general partner, for any chore we choose, what happens to our old economic relationships?

The magic genie of AI may not be a “normal technology” (in the sense that Arvind Narayanan and Sayash Kapoor describe), but because it is real, it will have limits and costs. It will never be costless. Some chores will cost more than others; some will be easy for one AI and near impossible for another. Professionals don’t just do better work than amateurs; they are often 10 or 100 times faster. Magic genies will show the same spread. One might speed through task A but stall on task B, while another does the reverse.

So immediately, we might go to other genies because they have specialized in some dimension. They might be cheaper, faster, or smarter in some way. Our own genie will provide what we ask, but at a greater cost, or a longer time, or inferior quality. The giant brains in the sky that serve 8 billion people will deliver abundant content, and abundant sameness.

Difference becomes the valuable scarcity.

The value will lie in doing things in a way that the giant brain in the sky does not. As AIs become cheaper to train, they will become easier to personalize, and we could have 8 billion slightly different AIs. But cheap variation is not the same as valuable difference. Personalization alone will produce endless variety. What yields scarcity is a distinction worth paying attention to.

A more important reason to rely on other genies is to collaborate. As smart as an AI is, it will never be as smart as all AIs together. It’s not hard to imagine large, complex projects that require hundreds, if not millions, of AI genies working together to create something astounding. Some of those AIs may have bodies in the form of robots, but the necessity of collaboration is still present. Because AIs will be plural and differentiated, the collective of them offers abilities, skills, and powers not present in lone individual AIs. As is typical of hives, colonies, and swarms, new powers can emerge from them that are beyond the sum of the powers of their parts. This is another attribute that will differentiate AIs: some will be better at collaborating than others. That sociality will make them valuable. (See my essay “Swarm Intelligence.“)

Imagine two ace personal assistants. One is super dedicated to its boss; it is the boss’s exoself. It has been honed over decades to serve one person superbly, and nothing else. The other personal assistant has similar dedication to one boss, but also works super well with others, and is capable of collaborating with almost any other AI. The second is more valuable, because it can bring the abilities of every other genie to bear on its boss’s problems, not just its own. By dint of both training and experience, not all genies will be the same, so we should expect a market for genie skills to arise.

If value lies in genies that are different and that work well together, then whatever passes between them becomes the thing of value. So the next question is: how is a genie’s magic transferred? Do you just share the magic spell that releases the good stuff? Is the genie itself portable? What about the personal context that makes the work so valuable?

Tim O’Reilly hosts Foo Camp, where alpha geeks converge to regale each other with their latest insights. The most recent Foo Camp focused on AI, and at this meeting Tim asked the group a profound question: In an era of AI, what is the shareable artifact of knowledge? He followed up with an essay called “The New Knowledge Artifacts of the AI Economy.” It’s focused on software developers, but as Tim notes, everyone is now a software developer and has a stake in this question.

In the past, books were the primary unit of knowledge. When computers came along, knowledge was combined with action, and the shareable unit became software code. The web made the page and the link the unit, and for twenty years knowledge spread hyperlink by hyperlink. Code itself became shareable as packages and libraries that anyone could snap together. Later, mobile phones refined the knowledge unit into the “app.” Tremendous amounts of potential knowledge and action are compressed into a tiny app, which can be shared, bought, and sold.

Presently, it does not seem that all the knowledge value of AIs can be compressed into an app. As Tim points out, the code of an app is missing all the rich context that informs the actions an AI takes. Let’s say you ask your AI genie to create a program to track your medical records in real time, schedule regular tests, and coordinate with your human doctor. It then suggests courses of action that your medical team (doctor + AI) recommends. A friend of yours hears about how much you get out of it, and asks if you can share it with them. Handing over the text prompts to input into another AI does not capture all the power your genie provides you after years and years of training on your life. Perhaps all that context can be packaged up into a bundle with the prompts, but if you pass that bundle on, it omits the particular circumstances of your friend’s life. The same is true if you hand over the genie itself; since its weights are tuned to you: it fits your life, not theirs.

Now that prompting a genie in everyday language is so easy, what gets shared will be the specifications – the goals. The value is in getting the question right. The spec is like a consultation. Like a seed it grows differently in different soil. My friend’s genie holds a consultation about my friends goals, and from that conversation it grows its own prompts and then actions. At some point, a protocol will emerge that produces specs for genies in a uniform enough way that they can be shared widely across many different environments.

An adjacent challenge is how to engineer the shared artifact that passes from agent to agent. A growing share of the world’s financial transactions will soon be between one AI agent and another. My magic genie will be hiring your magic genie for certain tasks (because they are done either better or differently), and it is not clear yet what will be passed between them. Sometimes one agent can give a prompt, sometimes a prompt with all the background files, or it might need to deliver a bundle of prompts, plus files, plus context, plus specs. Someday entire models might be portable and you hand over weights. Other times, the deliverable might be simply an action: my genie confirms a delivery, and that triggers your genie to send the invoice. In all cases, the packaging of this “knowledge” will need to be routine and standardized. Early versions are already here, such as Anthropic’s Model Context Protocol, and Google’s Agent2Agent protocol. Also “skills,” which are folders of instructions and files an agent loads when needed. And because genies will be hiring strangers, these packages will also need to carry their provenance, so a genie can tell whether a bundle is good, safe, and from whom it claims. (See my Trust Quotient)

Standardization does not preclude difference and variety, which will become the precious scarcities. Standardization is what makes difference valuable. The standard 26 letters of the alphabet, and our standard dictionary of words, enable the value of weird arrangements of letters and words to be captured by anyone. Just as we humans can produce endless abundant strings of words, but only a few distinct ones are worth paying attention to, our magic genies will endlessly produce movies, software programs, simulations, entire worlds, and we all (agents too) will gravitate to those that are distinctive in some manner.

Which brings us back to the first question. We will still need other people in so far as they can lean into their particular life experiences, and together with their personal genie, generate distinctive magic.

The new artifacts of knowledge will succeed if they can carry those distinctions in a standardized way, the way 26 identical letters, the same for everyone, carry every distinct thing ever written.

• Another reason it’s unlikely there’ll ever be a third generation Martian native: Dust. Fine and ultrafine Martian dust presents additional risks for future crewed missions to Mars
• I love Bob Dylan, in part because he’s from another planet and doing his own thing. Pope Leo receives Bob Dylan at the Vatican

• We now have a genuine world model in Google’s Genie4. Real time generation, volumetric, 3D navigation, language prompts. Awesome! Genie 4: A new generation of world simulation

Every young person between the ages of 18 and 21 should provide 2 years of mandatory national service. Accepted service goes beyond military service and includes civilian service like teaching, or construction, or emergency response. Everyone serves. There are no exemptions for wealth or connections, or even handicaps; service would be matched to ability, not waived. This would be good for them, good for their country, and good for the world.

There are several main benefits:

• This duty partially offsets some of the rights citizens enjoy.
• Useful societal work is gained that may not be as well served in the marketplace.
• There is a meaningful mixing of people from diverse backgrounds, helping political unity.
• Kids get a gap year, in the work space, that makes them better students in higher ed.
• They may get training, but surely get a sense of service and accomplishment.

I am an American, so I will make this argument primarily in American terms, but examples of this exist in part in other countries of the world.

The Work

To satisfy the requirement, a young person can join the military, in all branches. They would get training in a skill, and be under military command. With the advent of such a program there could be an expansion of forces, such as coast guard, space force, and cyber force to accommodate and harness youthful energy and enthusiasm.

Civilian tasks could include emergency response work, helping out in the inevitable natural disasters, from providing food and health care, to construction of temporary housing.

Outdoor labor in our parks and vast government lands is badly needed. Much of the western US is owned by the government. Fuel reduction to reduce wild fires is long overdue. Like the CCC in the past, crews of young people could be clearing brush, making trails, serving in state and national parks.

The best way to learn is to teach. Recent high school graduates have much to teach younger kids. Tutoring is underappreciated and very effective. Teaching is needed in rural and city, and is also helpful overseas. An expansion of the Peace Corps is also national service.

Not all jobs would be manual labor. A great benefit to the country would be to scan, digitize, train, and organize historical data from local municipalities, overlooked archives, city governments, etc. This data is useful on its own and for model training towards a public AI. There is also the need to monitor, track, catalog wildlife species and ecosystems in order to manage our lands.

Public health outreach and education is also badly needed and can be helped by newly trained volunteers.

The Payoff

Mandatory service when you are 18-21 creates a productive gap in education that comes at the right time. Many college teachers observe that kids who return after a gap year or two are more motivated, more committed, and more likely to graduate than those who come directly from high school. The break gives perspective, and the experience in the world gives reasons and directions to pursue higher learning.

The jobs that are outdoors allow kids to get out of their heads and off their screens. Their bodies respond, and their minds follow. It’s not uncommon that people bond to the places they work in. The parks and lands benefit later as those kids become adults who still care about the land.

There may be nothing as effective in countering political polarization as working with people from vastly different childhoods. Neither colleges, nor workplaces, would mix people more thoroughly as mandatory service (no exceptions) would. The result would be life long friendships and admiration between adults who otherwise would not engage with each other.

The Costs

The main drawback of this scheme is the cost. Today there are about 18 million 18-21 year olds in the US. With each serving two years, about 9 million would be in service at any time. At roughly $50,000 each, that’s about $450 billion a year, about 60% of what the US spends on all military compensation, including veterans’ benefits. Giving them all a job would be like hiring 5% of the nation’s workforce.

Notably, some of these outdoor, healthcare, and data collecting tasks will likely be among the last jobs to be replaced by AI, so I would argue they will be well worth paying for.

There are other costs: corps labor would compete with paid workers, and compulsory civilian service would face a constitutional test the military draft never did. Both challenges are solvable, but real.

About 60 countries have some form of mandatory military service, mostly for males. Fewer than half offer a civilian option, usually structured as a conscientious objector alternative, a sort of second-best choice, and often deliberately longer than military service. I am aware of no country today that requires universal service of all young men and women with civilian options as equals.

I think the value of such a program would immediately pay off in better education results, less polarity, and more committed citizens, and it would compound over the years.

• The internet archive’s Wayback Machine library is a planetary treasure. It preserves the archive of the web. Don’t let publishers take it down. Why the Internet Needs a Memory
• PDOOM is a great music video slopcore with a very SILO like ending. These are getting so good. Astra Blue AI – P Doom (Official Music Video)

It is commonly said that AI today is the worst it will ever be. This is true. AI today is probably the most centralized it will ever be as well. Currently the most reliable way we know how to make AIs smarter is to make them bigger. This principle is called the scaling law. To improve the current best AIs, which are transformer neural nets, we need to increase the scale of their training data, the scale of their compute, and the scale of their inference, or delivery. That scaling results in massive computer processing centers, running millions of chips, with trillions of parameters, powered by gigawatts of energy, costing tens of billions of dollars. Very few can afford this scale of system, so this AI power is concentrated into a very few organizations.

On the other hand, each one of us runs a very powerful (better than most AIs), very tiny (big as your skull), very energy efficient supercomputer running only on 25 watts at the end of our necks. There are some 8 billion of these tiny supercomputers spread around the planet, interconnected via the internet. We have an existence proof that biological intelligence does not need to be centralized.

Today, because of rapid innovation in AI architectures and training formulas, the amount of compute an AI needs to deliver a given capability has been cut roughly in half every 8 months. Combine that with the hardware gains of Moore’s Law, and the cost per capability halves roughly every 6–7 months. Assuming Moore’s Law continues, within a few years, a model as capable as today’s leading LLM – let’s call it Opus – could run on a device you own. Within about a decade, the cost of training one from scratch would be affordable by an individual, and fine-tuning your own copy would cost far less.

That means everyone could own and control an AI like today’s Claude or ChatGPT or Gemini. You would not have to ask permission to do things, or be obligated to train it with certain guidelines, nor have to accommodate the preferences and sensitivities of others. The AI could reflect you.

This kind of shrinkage should not surprise us. It seems insane that a multibillion-dollar data center occupying a vast brown field could be reduced to an orb that sits in your hand, but consider what has already happened inside your phone. Twenty-five years ago, the graphics chip alone in today’s iPhone would have filled a large room. Your phone’s AI chip would have required acres of computer cabinets. It is likely that the entire controversy around massive data centers disappears because they disappear; they were a temporary scaffolding needed for the first round of tech, just a blip in the history of technology.

Of course in 10 years, you would be stuck with today’s AI – which as I began – is the worst it will ever be. But at some point in the very near future the amount of AI that you need for 99% of most jobs could be provided by a local device. It is similar to bandwidth. At a certain point, most tasks you have can be done with a gigabit connection. And once you have a gigabit connection, you are happy. Maybe when we have the metaverse, or the mirrorworld, or augmented reality, we’ll elevate that minimum, but the point is that it will be good enough. At some point the smartness of consumer AI will be good enough for 99% of what you need, and that smartness will be able to run in something you own and could also train.

Many of the issues that people have with AI today would be resolved if the AI were not only available from a huge massive monopolistic tech company, but instead was generated, delivered, overseen, and maintained by individuals. A lot of the criticism of AI today is really just a criticism of capitalism. There is the question of who benefits from the value being created, who controls who gets what (the best AI models may be kept for internal use by some companies), who gets to train the AI mind, and supervise what it is trained on, and who decides what it is used for.

Not all issues are resolved by individualized AIs (they will still have tricky trust issues), and some new problems are created. But I find a vision of a billion AIs running the world much preferable to 3 or 4 of them running the world. Decentralization spreads power, but more importantly it can deliver something a central AI never can: an AI as unique as you. Here is my ideal hope:

I hope that every person born (and yet unborn) can be on a path through their lives to become the most improbable version of themselves. (See my manifesto Your Most Improbable Life). The idea is that you become unique, with a unique set of talents and experiences, occupying a niche in the world that you create. You slowly invent your job. You are not the best, you are the ONLY.

This kind of self fulfillment is possible because we have technological agents that relieve us of the tasks anyone can do, leaving us with the tasks only we can do. While there will be billions of ordinary and commodified agents (bots and robots) that perform these everyday tasks, the best of the AIs will be those we tailor to ourselves.

I want a personal AI that I have trained myself on all my life’s experiences, all the influences I have had, all the work I have done, and all the people I most admire. There will be advice and schools for training your personal AI to become the most useful to you. In fact there will be teacher AIs who educate personal AIs, and people will hold opinions on the best way to educate your personal AI and what kind of AI teachers you want. You want the teachers weird but not too weird. You may want to train your personal AI on the classics, rather than everything. The current frontier models are trained on everything and they are aimed at everyone for all problems. They are by default, generalists hitting the average in the middle of the mainstream.

But we want a personal AI, an exoself, that is as unique as we are. We don’t want the best AI, we want the ONLY AI. We want an AI that is as improbable as we are, and improbable in the same unique direction that we lean. We’ll want to curate the education of our AI.

At least I do. I want to work with my exoself AI agent to co-create, co-manage, co-invent my life. I don’t want to use the same AI that 8 billion other people use. How can that help me to be unique? In order to be creative, my AI exoself also needs to reflect my own uniqueness, doing its own AI thing with distinction. It should be as weird as I am.

The art of training your own personal AI to be weird yet useful will become one of life’s skills. An entire section of YouTube and self-help programs will be dedicated to cultivating the Agent of You. The ultimate goal of this Agent of You is to help make you the most improbably unique person you can be.

Mainstream will never go away. And for much of life the ordinary is more than fine. There may be big-tech janitorial agents, and brand name clerical bots, and exclusive alien genius bots, but the AIs that can serve creators should be as weird and personal as we are.

Here is a possible world 50 years in the future, one I think is more likely than the scenario most people expect, which is one huge AI in the sky.

Fifty years from now most AI is invisible. There is nothing to see. It is sort of like air – always there, always on, everywhere. Something you need and use but take for granted. AI is so prevalent we become blind to it.

You don’t need to log onto it, or even to summon it. Rather you inhale it. Instead of a central AI brain, AI is like a gas, composed of zillions of tiny interacting AIs that create an atmosphere. These zillions of minds are like eukaryotic cells in your body, or hundreds of varieties of bacteria in your gut. They form a presence. They are woven together into a vast network of intelligences that create an AI background. This society of AI minds is not so much seen by humans, as it is felt.

Unlike the AI of today, it will be deeply embodied – into robots, vehicles, buildings, and devices. But its embodiment will be so wide, so deep, so pervasive, that it will become the background.

AI might be thought of as a superorganism that is so vast that, like an ocean, it becomes the environment. This environmental AI, like many environments, can become imperceptible to those immersed in it. We’ll only notice it when there is a gap, a disconnection, or it goes down.

Like a coral reef, an AI superorganism will be composed of thousands of species of minds.

Some of these AI minds might be very narrow and dim, just a notch above being stupid, maybe as smart as an insect. Others might be math geniuses. Most sub-AIs will be engineered in the middle to do specific boring and niche tasks.

The chief characteristic of any superorganism is that its power transcends the power of its parts. So the superorganism of AI, this AI environment, will transcend the power of its parts – which will include all the frontier models of the time, and the most powerful specialized AIs for math, or biology, or world building, and the general purpose everyday AIs running the back offices of corporations. Interacting together, these AIs will form a system of co-dependency, with different AIs relying on each other for inputs, guidance, maintenance, support, direction, amplification, resolution, upgrading, verification and trust. This ecosystem of AIs will be the environment, and just as in organic life, each AI will co-evolve with this environment, this society of minds.

Here are some notes on what we could expect for this kind of swarm intelligence.

• Maintaining a sustainable swarm intelligence is tricky. The science of complex adaptive systems indicates that there are sweet spots in the connectivity between swarm parts. Parts that are too heavily interconnected seize up in rigid paralysis; parts too thinly connected fly apart in chaos. Tweaking systems so that they don’t collapse but thrive and flourish is almost more art than science. A sudden collapse of a swarm intelligence will become a prime danger while using them.
• AI protocols will appear. More than APIs, these standards will enable unlike minds – dumb ones and super smart ones – to negotiate, hand off, trust, earn a reputation, and store memory between themselves. These all operate in the background; no ordinary human thinks of them.
• No single institution, corporation, or government will be able to control the swarm environment. When a networked mind makes a mistake, there probably won’t be a single author, owner, or operator to blame. While some institutions may control pressure points, or critical protocols, or bottleneck infrastructure, no institution will be responsible for what happens when people use AI. It will be similar to an internet for AI, which will have multiple stakeholders that keep it going. This diffuse liability is both a blessing and a curse.
• A distributed liability will likely birth new kinds of distributed and decentralized insurance and adjacent law. In addition, the need for ways to verify and trust trillions of agents, all claiming good intentions and competency, will require a whole new technological trust system that can verify, authenticate, evaluate, and rate unlimited alien agents and sub-minds. A significant cost of transactions between parts of the swarm intelligence will be dedicated to the trust network.
• We currently don’t have practical ways to assign accountability when harm emerges from interactions between AIs rather than from a single bad actor. We will need metrics to trace influence across networks, to estimate counterfactuals (“what would have happened without this component?”), in order to govern the swarm.
• A society of minds will produce parasites, scams, slop, spam, and adversarial hacker agents. Most of the agents in the swarm will be actual bots. We’ll grow some equivalent of an immune system with white blood cells to detect, isolate, and diminish the harm. Creating the immune system will be an economic opportunity for many.
• As a superorganism, AI will keep producing emergent phenomena and behaviors. It will generate varieties of thinking that are not present in any of the component AIs. This will be worrisome and exhilarating.
• Because of the planetary scale of this superorganism at its largest, much of its behavior and intelligence will feel alien. Some may call it inhuman, but it will certainly be unhuman.
• There will be multiple levels of swarm intelligence. One level will be planetary. Just like a body with tissue, there will also be sub-levels of swarms operating at the level of nations, and certainly corporations. In fact, as corporations become more and more cognified, permeated with AI, they will more and more resemble outright AI swarm intelligences. Another way of saying this is that more and more swarm intelligences will begin to do the work of, and act like, corporations.
• Rather than engineering, the best way of managing swarm AI may be more like gardening. We’ll use containment, many layers of fences, permissions, borders, and isolation in order to nurture the results we want. We will plant many species of minds, shape the environment, and cultivate the ones that behave well. Instead of controlling the system by understanding every internal mechanism with one clever algorithm, we’ll steer the way we govern markets, cities, or any system: indirectly, imperfectly, with observation and constant intervention.

So an average day 50 years from now in this swarm AI world, we will not be talking about AI. We will hardly see it. It will be everywhere, embedded into everything. We will have one or two personal agents we depend on and like; these exoselves might be like our twins or partners, but beyond them, we won’t be encountering AI. We’ll just encounter the world – cars, tools, kitchens, planes, offices, doors, screens – and that world has AI embedded into every bit of it. Behind our view, an ecosystem of AIs interact and are co-dependent on each other, animate the world with pervasive cognition, such that it is hard to determine where one mind starts and another ends. The environment is smarter. We will still carry our magic stones and wear smart glasses, but most of the intelligence will be embedded into the background. Getting this swarm intelligence to work will be the job of the next generation of inventors.

• Not a normal debate. The debate about whether AI is a “normal technology” has been spun into overdrive with AI-generated musicals defending each side. (AI-musical is becoming a thing.) A Normal Debate?

Earlier this year I made a bet with Craig Mod. He felt sure that robots would soon be able to play basketball better than humans. It’s a reasonable prediction based on the fast advancements in robots running sprints, kicking balls, and shooting baskets. If they can beat the fastest human in the 100 meter dash, why not trounce the Warriors? I think it is inevitable that at some point, we could make a specialty machine that would play basketball better than any human could, but I am skeptical it will happen soon. Craig claimed that by 2040, robots could beat the best human basketball players in a full regulation game. I bet against this timeline and say they will not. The winner has to buy a round of gelato.

I am incredibly optimistic about AIs. They will not only do a lot of knowledge work, and change our jobs, they will also accelerate most scientific and technical fields – including robotics. Faster AI models will inhabit the minds of robots and will also enable our minds to figure out all the technical problems humanoid robots present. There are many.

The major challenge is cramming all the necessary power and compute into a mobile unit able to perform for several hours. It turns out that our form factor is incredibly efficient. Compared to what we do with machines, we are insanely efficient. Our supercomputer brains need only 25 watts, which is orders smaller than LLMs. At their max our human bodies generate less than 100 watts while walking, maybe 400 watts playing basketball, while machines need about 500 watts to walk, and 2,000 to exert strenuously. Gaining five fold efficiency in motors is not impossible in 15 years, but we have been actively optimising motors for over 100 years, so this is not a new incentive. It will be difficult.

In addition to our incredible energy efficiency, humans have two other advantages at the moment. Our resilience and recovery. Robots are stiff, and hard, which means when they fall, they break. Basketball is a contact sport. If a 7 foot human pushes over a robot, it may never get up again. Of course the inverse is true too. Getting hit by a machine will seriously hurt. However humans have a remarkable ability to recover, to get around an injury, in a way machines currently do not. Very small injuries can incapacitate a machine. We don’t have a good way to overcome that yet.

The most important hurdle basketball-robot designers face is the flexibility of humans. It is not that difficult to create a machine that is excellent in shooting baskets, or excellent in sprinting, or excellent in jumping, or excellent in dribbling. The challenge is to combine all these abilities without too many tradeoffs. Humans obviously are an existence proof that all these capabilities can be tuned, but at the same time basketball is a sport that is optimized for human skills that come from water-based meat tissue. Trying to optimize silicon and titanium on the same set of meat skills is a serious hurdle.

The terms of the bet require a full team of robots no bigger than professional basketball players, playing with standard rules of substitutions and duration, competing against the very best pro human players. I might concede that a basketball robot could beat an average human by 2040, or even a decent high school team, but the level of athleticism in professional basketball players is superhuman. They too are alien beings. And very quick learners. Getting the best of them in a sport they live and die in is a near-impossibility by 2040.

Any success in a basketball robot will come from a robot that is designed specifically to play basketball. It will not be able to win at volleyball, let alone football or baseball. The greater lesson I think we will learn is that a general humanoid robot is not going to be that useful. Rather we’ll construct a lot of specialized robots to get things done. Retail store restocking robots, cleaning bots, cooking bots, hospice robots. And in sports, the best performers will specialize in a way that humans kind of do.

To be very clear, I fully expect to watch robot olympics. I will be one of many willing to pay to see robots dance, compete in sports, and amaze us with their kinetic abilities. Basketball robots are likely to invent new basketball moves that humans didn’t think of doing, or couldn’t do easily. In the long-term, robots could change how humans play basketball professionally.

I have no doubt this roboty world will happen, I just seriously doubt it will happen in 14 years. I can taste the gelato.

I made some notes on the nature of the training of LLMs, and about whether we as a society should consider the material used to train them as a fair use of that material. I believe it would be best for us to consider them fair use, and I made some points in its favor. The context of these points is US copyright law, which I did not spell out here. There are other commonly used arguments pro, and many arguments against, which I have also not listed. I wrote these points as a way for me to think aloud along my own lines, and they are not arranged in any order to make a tight case. I welcome constructive comments, or points I may have missed.

1. Text, images, or music, etc, are all expressions made by humans, which are copyrightable. During training LLM transforms these expressions into an abstraction called “latent space.” It takes one kind of thing – expressed content – and transforms into another kind of thing – an almost mathematical abstraction that contains no expression. This is why LLMs are called transformers. The latent spaces of an LLM are closer to something like a syntax which is not copyrightable.
2. This latent space transformation is a fundamental transformation, because it goes one-way. It is an asymmetrical process: Content can be moved into latent space, but the latent space cannot be reversed to go back into the original content. It has truly been transformed.
3. The usage patterns of how the general public uses the transformed material is new. When an LLM is asked to troubleshoot a broken electronic device, this service does not resemble reading a book or a newspaper article. When an LLM is used to develop a marketing strategy, it is a profoundly different action than reading text. When an LLM is assigned the task of proving a math problem, it is not substituting for a book about math. The transformations accomplished by LLMs are both in their form and in their use. Not only are they transformed, but they are transformed into a whole new category we have not made before.
4. The key benefit of LLMs is their smartness, which emerges from the latent space and is not present in the source material itself. Their ability to pass a history exam is not present in any history book, and is not even present in ALL history books. These new benefits emerge from the process of creating an LLM trained on history books, and then transformed into a latent space, which produces this smartness. Much, if not most, of the value of LLMs derives from their ability to generate new benefits beyond what the source material alone can yield.
5. The information inside an LLM is not stored in the form of copies of things. Even though an LLM may know the full content of millions of books, it does not contain within it any copies of the books. It may be able to recognize any object in a picture, even though it stores no pictures. Instead of containing a copy of the image, it contains the information contained in the image. The weights of an LLM can be copied, but the latent space itself does not resemble a copy. The latent space is a non-copy entity.
6. Intermediate, transitional copies are the norm in the digital world. When your phone pulls up a web page it is technically making a copy of that page for a brief moment. When you send an email, it is copied in transit by telecom companies many times, but we don’t count them as invoking copyright. Since these intermediate copies are not stored, we don’t constrain them. Courts have already permitted literal copying that never surfaces publicly, The copies that LLM’s read once during training, are not stored, are not surfaced, and are therefore normal transitional copies that are fair use.
7. The genius of the LLMs comes in part from the astounding fact that all bits of information from millions of different kinds of sources and subjects are mapped onto a single “map”. There is a single conceptual space (the latent space) where every fictional story, and every bit of medical information, is combined with all geological knowledge, and all news events. And so on. As a result the value of any particular contribution to the overall value of the LLM is almost incalculably small.
8. Any individual item used in training the LLM has a very low value because 99% of almost every source is redundant. There is very little unique information in any item, even for seemingly original material. When an additional piece of content is inserted into the conceptual space of all human knowledge, most of it is redundant with other material. That means, removing an average bit of source content would make virtually no difference to the output of the LLM. As LLMs continue to scale up, the value of any particular source continues to diminish.
9. Modern LLMs have trillions of parameters – that is they have trillions of attributes they are describing. The consequence of mapping all human knowledge into one latent space with trillions of attributes is that every bit of information intersects, or influences, every other bit of information. Calculating this enormously complex relationship between trillions of influences is the grand task of a massive data center. This calculation is the most complex computing task we have ever done. But it is so complex that we cannot practically unravel the ripple of influences of any particular source. The degree of influence for each source content is therefore not calculable at this scale. It is not dissimilar to our considerable inability to calculate in a quantifiable way the influences in our own thinking and knowledge. Therefore the import of a particular source cannot be practically measured. 
10. In creating its knowledge, an LLM does not distinguish between sources of influence in its training. There are more hours of Star Wars reviews than Star Wars movies. You will have a lot of knowledge about Star Wars even if you have never watched a copy of a single one. What an LLM knows about Star Wars may come primarily from sources other than the movies themselves, from authors who have watched Star Wars. These diverse secondary and tertiary influences are doing a lot (most?) of the work that LLM delivers. Verbatim quotes can even come from secondary sources, which themselves are using the primary material in a fair use way. In fact the high value of LLMs derives in large part because indirect sources of material give a necessary and “intelligent” context to its knowledge. In this way, the LLMs improve the primary source material, to make it truly useful. This is another way LLMs transform material.
11. When a human student is studying, they might learn from a book they purchased or from a book they borrowed from a library, which we consider fair use. We don’t judge their learning based on whether they paid for the source, or whether the sources were borrowed or rented. Likewise if an LLM is trained on borrowed copies from a library (fair use) that should not impact our judgement of its knowledge.
12. The constitutional rationale for US copyright is incentivizing creation for public benefit, not necessarily securing revenue streams. I’ve personally seen anecdotal evidence that LLMs have been encouraging massive degrees of co-creation. Nearly every day someone sends me something that they have recently created with the aid of an LLM. You might dismiss this as slop. Most of it is. But most of what humans traditionally made everyday could also be called slop. And the best of the slop will get better. What is clear, is that the public benefit of transforming expressions into latent space is huge, and should be encouraged.
13. At the moment the smartest AIs humans have invented are LLMs, but that might not be true in the future. There are lots of alternative models to LLMs being experimented with, and they could become the default models next. Today the most advanced LLM models are intensely commercial (and rich). But they might not always be. We already have various types of open weights, open source LLMs. We tend to think of LLMs as chiefly commercial entities, trying to maximize revenue. But LLMs don’t need to be profit-making. Many of the open source models are matching the capabilities of for-profit models today. LLMs can be non-profit. They can also be considered a true common wealth, like a public domain, or like the internet. I call one version of this kind of AI,  Public Intelligence. There could be more than one public intelligence. These LLMs would have multiple stakeholders, be publicly funded and publicly accountable. They could be trained on all texts in all languages from all times. In part they would be like a super library with a super librarian, able to do most knowledge work. This public intelligence would be available to anyone, at cost. The latent spaces of the LLMs (whether profit or non-profit) are our new public domain. The purpose of copyright is to maximize the public domain by encouraging abundance of creations with temporary monopolies and then moving the protected into the commons as soon as possible to assist in further creation. The new LLMs are a new kind of commons that provide society with new powers of understanding (getting fantastic answers to questions), new ways of self-improvement, new ways to collaborate, new ways to be creative. It would be best for society and everyone born and yet unborn, if public LLM commonwealths were encouraged. We should maintain the fair use option for training LLMs to ensure we have at least some public intelligence commons.