I’ve recently wanted to delve more into history behind the current AI boom because there’s so much noise right now, and this noise obscures the reality about how we got here.
I ran across Karen Hao’s book Empire of AI: Dreams and Nightmares in Sam Altman’s OpenAI at the library and picked it up on a whim. I’m glad I did. It’s a hefty and comprehensive book, coming in at over 400 pages, that provides a wide-ranging survey of the the beginnings and main forces at play within the modern AI industry, even if it’s missing some of the most recent developments (it was published in the “ancient” time of January 2025, and ends with OpenAI as the market leader whereas today Anthropic has raced ahead). I don’t want to do a thorough review of the book, but I do want to highlight some key points she covers that I found fascinating.
Much of the book– as the title suggests – focuses on the complexities of Sam Altman as a character, particularly his duplicity. I won’t cover any of this since this has been a well covered topic in the news.
How OpenAI’s founders came together Link to heading
The seeds of OpenAI were sown in Elon Musk’s concerns over AI, especially in his early conversations with Demis Hassabis, the founder of DeepMind (which at that time had not yet been bought by Google). Musk invested $5M in DeepMind to “keep tabs on the company.” Musk then had a big argument at his 2013 birthday party with Google cofounder Larry Page, who didn’t believe that AI surpassing human intelligence was a problem. These conversations started Musk’s tour to tout the dangers of AI.
When Google bought DeepMind in late 2013, this motivated Musk to explore ways to counter Google, as he believed this was a bad combination and could lead to AGI (artificial general intelligence) harming the world as Google sought to use it to maximize profits. This also began Musk’s characterization in public of Hassabis as a supervillian.
Sam Altman began communicating with Musk in 2015 over their shared concerns about AGI. Eventually, Altman proposed that they start a “Manhattan project for AI” that allows the technology to be owned by the world via a nonprofit. Musk agreed to the broad outlines of Altman’s proposal, and Altman then went into recruitment mode, bringing together the top engineers and AI researchers he could think of to a dinner with him and Musk at one of Musk’s favorite restaurants. Greg Brockman, Ilya Sutskever, and Dario Amodei were among this group.
The first person to commit to OpenAI was Greg Brockman, a Harvard/MIT dropout who was the fourth person at Stripe and known as a “10X engineer,” later becoming Stripe’s CTO (particularly impressive given that Stripe’s founders themselves were top notch coders from those universities themselves). While Brockman was not an AI expert himself, he sincerely believed that AGI could be around the corner with enough effort and focus, and this belief was strengthened by experts like Sutskever being open to the idea of developing AGI at the initiative, even if there were no commitments. Altman then played matchmaker to him and Sutskever, a deep learning wunderkind who, with another graduate student Alex Krizhevsky in Geoffrey Hinton’s lab at the University of Toronto, had created AlexNet, a groundbreaking deep learning image recognition system. After AlexNet, Hinton subsequently created a shell company comprising those three people, which they sold to Google for $44M. This meant that Sutskever at that time was working for Google, and Brockman took it upon himself to be the recruiter, trying to convince him to leave and join this new initiative instead.
Brockman also networked with other leading figures in the AI field including university professors to identify the top AI students and researchers. After several weeks, his single-minded dedication to recruitment paid off, as almost all his targets accepted his offer to join OpenAI. Hence the “brain trust” of OpenAI was formed with Altman, Brockman, and Musk leading the initiative. An interesting tidbit mentioned in the book is that when Google found out Sutskever was leaving to join OpenAI (with a whopping salary of $2M), it offered him 2-3x that amount to try to retain him. But this only convinced Sutskever that an organization like OpenAI was needed.
OpenAI’s structure and mission commitments were by design unclear Link to heading
Altman’s initial proposal at OpenAI’s founding was as a nonprofit focused on “openness.” Openness in those early proposals involved open sourcing the science and transparency, without allowing any specific company to control AI.
However, in later correspondence between Musk, Altman, and Brockman they acknowledged that they could walk back their commitments to openness as necessary to avoid bad actors getting their hands on AGI technology. So the seeds for changing OpenAI’s structure and charter were sown right at its start.
AI is now synonymous with deep learning, but it wasn’t always so Link to heading
Hao provides a brief history of AI to show that its definition and status has constantly been changed, depending on who was best able to market and own the term.
In AI’s early days, meaning the 1960s and 1970s, the fight was between the “symbolist” and “constructionist” camps. The former believed that intelligence comes from knowing, and hence AI could only be created using logic that could be reasoned about. The latter believed that intelligence comes from learning even if one couldn’t fully encode the reasoning behind everything, and hence AI depends on creating machine learning systems, such as by mimicking how the brain processes information. This battle played out primarily in academic circles. The symbolists, led by MIT professor Marvin Minsky and his colleage Joseph Weizenbaum, eventually “won,” bolstered by Minsky’s withering critique of connectionism and neural networks in his 1969 book Perceptrons and Weizenbaum’s successes with a symbolic chatbot called ELIZA. Neural networks were then considered a fringe subject for many decades until the 2010s, leaving only a small set of true believers fighting for scraps of funding to continue that research, including professors Geoffrey Hinton, Yann LeCunn, and Yoshua Bengio.
The term “deep learning” was coined by Geoffrey Hinton as a shorthand for using deep (multilayer) neural networks to perform machine learning. With the success of deep learning in speech and image recognition shown by Hinton’s lab and other researchers in the early 2010s, large tech companies including IBM, Microsoft, and Google clamored to put such technology into commercial applications. Google was the most aggressive to do this, and it found success with things like improving Android’s speech recognition. Most importantly, Google’s business model was based on collecting and utilizing large amounts of data, which was a perfect fit for the needs and applications of deep learning. The focus on collecting ever more data that in turn improved deep learning models led to its dominance as the primary form of “AI” in the next decade.
On the other hand, the failure of IBM to commercialize Watson, it’s symbolic AI-based system that was great at Jeapordy!, further showed the weakness of the symbolism approach in modern times.
The numerous factions within the modern AI movement and their beliefs Link to heading
OpenAI’s founders and early team came from a mix of several philosophical groups:
- Effective Altruism (EA): This philosophy, popular in Silicon Valley during the early 2010s, believed that one could make the world a better place through logic and discipline, essentially combining capitalism and libertarianism for moral purposes. One of its core believes is the concept of “expected value,” which can be calculated (abstractly usually) by multiplying the probability of its occurring with its quantified positive or negative impact. A logical conclusion of this belief championed by EA cofounder William MacAskill was that it was more altruistic in the long run to take a morally ambiguous job to get rich and then donate that money than it was to dedicate oneself to working at a charity and getting paid less. Taken to the extreme, this logic would manifest itself in Sam Bankman-Fried, the crypto billionaire from MIT who committed massive financial fraud while justifying it through the lens of EA. Over time through incidents like this and through reports of abuse and sexual harassment within the EA community, EA become a less popular philosophy (or at least people touted it less even if they privately still believed it).
- The AI safety contingent which included the Amodei siblings were part of this camp.
- The AI Safety community (or the “doomers”): these folks were focused on the risks of AI and viewed AI safety as a paramount concern and were content to limit AI adoption for the sake of safety. As I just mentioned, many of these people were also EA adherents.
- Effective accelerationism (e/acc or the “boomers”): This was a counter to the EA and AI safety movement, and its informal name was a joke against EA. These people believed that technological progress was universally good, and therefore it was a moral imperative to make technological progress as fast as possible on both AI development and AI adoption.
The primary drivers of OpenAI’s LLM advances Link to heading
Sutskever was not known as a coder, but he was known as someone who was able to intuitively grasp key insights that would lead to AI/deep learning breakthroughs. A year in, as OpenAI was floundering across multiple projects with no clear milestones or goals, Musk was getting impatient. Sutskever and Brockman therefore worked on developing a clearer research roadmap. Sutskever believed that advancements are directly due to the amount of “compute,” or computing resources, that OpenAI was able to utilize. While data quantity and algorithmic improvements were important, he honed in on compute as the number one priority.
This sparked the focus on “scaling laws” (must scale up computing resources, training data, and model size to be able to make significant AI advances), which in turn meant OpenAI needed tons more compute, which then meant they needed tons more money to acquire such compute, which in turn drove OpenAI to restructure to take investments to fund such compute resources. So it was interesting that Hao’s book highlighted that it was this raw acquisition of compute which drove OpenAI’s successes.
Additionally, developments in the field such as the Transformer architecture (which was ironically developed at Google but not capitalized on there) also enabled the LLM advances, and Sutskever was quick to realize the significance of this, despite skepticism of other researchers at OpenAI.
The consequence of scaling laws in particular is why we are still living the consequences of this, i.e., a world where energy and computing hardware are being gobbled up by ever increasing AI needs, leading to less and more expensive supply for everyone else.
Why the Amodei siblings were able to start Anthropic Link to heading
One thing I never understood was how the Amodei siblings with several other OpenAI defectors were able to succeed with Anthropic, given OpenAI seemed to have a massive head start and still the brain trust for LLM advancements. It was particularly strange to me given that Dario Amodei was on the AI safety team.
Hao’s book provides more details why.
Prior to joining OpenAI, Dario Amodei was working at Google, where he cowrote a foundational paper around AI safety. While he didn’t join OpenAI at its start, he kept in touch with OpenAI’s team and eventually joined several months later to lead AI safety research.
Dario Amodei’s AI safety team was focused on aligning AI systems to human preferences. With the release of GPT-1, Amodei decided to focus his group’s efforts on language models rather than the other toy environments they were previously experimenting with (e.g., video game virtual environments). Specifically, he wanted to support a model that allowed humans to interact with, such as a chatbot. He then joined forces with the lead of GPT-1 to build ever larger models to support this goal, with the idea that 3 key inputs determined the performance of deep learning models: the volume of training data, the amount of compute used to train a model, and the number of parameters. These were dubbed by his team as “scaling laws.” He carried these ideas to push forward on GPT-3, an even larger model requring ten thousan top of the line Nvidia V100s GPUs.
So Amodei was instrumental in pushing these models ever larger despite his primary role being AI safety. However, the power of these ever larger models and their propensity to spew venonous ideas was also the impetus for major disagreements between Amodei’s circle and Altman regarding when models should be released and what safeguards should be added before release. This spurred the breakup that led to Anthropic’s formation.
The OpenAI board was outplayed then left to hang Link to heading
The book also provides more details about the political infighting that ousted Altman, then reinstated him. Although always duplicitous, Altman had become even more so (and paranoid) as the company grew, leading to internal disfunction as different leaders were unclear who ran what, etc. A key disagreement was between Sutskever and another top researcher, which Altman didn’t resolve and instead seemed to fuel by giving conflicting answers that favored each side, depending on who he was talking to at any one time. CTO Mira Murati was often left trying to clean up internal messes as well as patching over overpromises he made to Microsoft.
Sutskever and Murati eventually both indepently approached Helen Toner, a young board member who was a researcher at Georgetown University. With her backing, the rest of the board decided to oust Altman for “untruthfulness.” They knew this was a vague answer, but they believed Sutskever and Murati (who the board decided would be interim CEO) truly wanted to remove Altman.
However, once Altman mobilized his powerful allies–which included Microsoft CEO Satya Nadella, Airbnb founder and CEO Brian Chesky, and LinkedIn founder Reid Hoffman–Sutskever and Murati switched sides and left the board hanging dry. In this, the book highlighted how the board overplayed its hand and thought it had allies which it didn’t, as well as how Sutskever and Murati didn’t stay loyal to the board despite being the ones instrumental in pushing it to oust Altman.