Early days of AI (and AI Hype Cycle)

I worked on early ML systems and products at Google and later at Twitter (after they bought my company, Mixer Labs). I then spent a decade working as a founder and executive & investing in machine learning companies. Until the rise of new AI architectures (in particular transformer-based and diffusion-model based approaches), roughly all machine learning startups failed. Value in prior AI waves went largely to incumbents over startups - as the capabilities were not advanced enough to create new market openings.

Here is a slide I used to use (borrowed from Brandon Ballinger) during 2017-2019 or so - this slide reflected the CNN/RNN/GAN world of the prior ML wave.

When many business people talk about “AI” today, they treat it as a continuum with past capabilities of the CNN/RNN/GAN world. In reality it is a step function in new capabilities and products enabled, and marks the dawn of a new era of tech.

It is almost like cars existed, and someone invented an airplane and said “an airplane is just another kind of car - but with wings” - instead of mentioning all the new use cases and impact to travel, logistics, defense, and other areas. The era of aviation would have kicked off, not the “era of even faster cars”.

(We should of course, fully recognize how important prior waves of ML and deep learning were and are to all this - however, to treat it as an ongoing continuum may miss the seismic nature of this technology shift).

Slide I now use.

The biggest inklings that something interesting was afoot came kicked with GPT-3 launching in June 2020. GPT-3 was a massive step up from GPT-2 and prior models. It was not quite good enough to do all the things we now view as hallmarks of “AI”, but it was highly suggestive of what was to come (I went on the A16Z podcast a few months later to talk about GPT-3, as it was so striking). For those in the know, the launch of GPT-3.5 in March 2022 solidified the perception of transformer-based models as the future. Internally at companies like Google, OpenAI, Microsoft, and Anthropic, early access to models gave a subset of people a glimpse of the future that was coming. This led to a Google engineer eventually proclaiming an internal AI chatbot named LaMDA as being “sentient” - this chatbot was a sort of predecessor to chatGPT and products like Character.AI.

The real starting guns for this AI wave in terms of a large number of founders jumping in was driven by two sets of launches. First were the launches of image-gen products like Midjourney and Stable Diffusion, followed a few months later by ChatGPT, which wowed the world, captured the public imagination, and was the AI startup big bang moment. ChatGPT truly highlighted the capabilities of these new forms of AI and the power of RLHF. OpenAI followed up with GPT-4, 4 months later.

True enterprise adoption is still many quarters/years away

ChatGPT’s launch was the starting gun for mainstreaming that AI is a big deal in terms of new capabilities and kicked off the large scale enthusiasm, hype cycle, and adoption for generative AI. This launch was only 8-9 months ago, and GPT-4 did not come out until 5 months ago. Given that large enterprise planning cycles often take 3-6 months, and then prototyping and building will take a year for a large company, we are still very far away from peak AI usage or peak AI hype. Most large enterprises are still trying to analytically sort what “AI” means for them, and are still many quarters from embracing this new technology.

4 Waves of AI Adoption

Indeed, there are likely at least 4 waves of AI to consider in these early days.

Wave 1: GenAI native companies. ChatGPT, Midjourney, Character.AI, Stable Diffusion, Github copilot, and other early launches that have now gained significant revenue and user traction. Obviously there are some great ML companies that pre-date GenAI that continue to participate in the current era (Hugging Face, Runway, Scale, WandB are a few that come to mind).

Wave 2 (current wave): Early startup adopters and fast mid-market incumbents. This is the first wave of startups to launch on top of GPT-3.5/4 like Perplexity, Langchain, Harvey or others. In parallel, a small number of founder led multi-billion companies like Navan, Notion, Quora, Replit, and Zapier launched AI-powered products quickly and are the early adopters of the wave. Microsoft, Adobe, and Google are notable outliers as very large enterprise moving fast to AI - Microsoft likely due to its inside track with OpenAI, and Adobe as diffusion models tend to be cheaper and simpler to train than the large scale LLMs.

Wave 3 (coming soon): Next wave of startups currently being founded. It will be exciting to see what is in this mix and may include new formats like voice and video in addition to using natural language in more verticals and more ways, as well as new types of infrastructure. Companies like Eleven Labs/LMNT/LFG Labs, Braintrust, and many more will provide incremental experiences. There is a big wave of new startups coming. The current YC batch alone appears to have a 100 or more AI startups….

Wave 4 (coming 2024/2025?): First big enterprise adopters. Since enterprise planning and build cycles are so long, anticipate the first really products (versus demos or prototypes) from larger companies other than MSFT, Adobe, Google, Meta to start to show up in a year or two. This is when revenue to AI infra companies will start to ramp significantly relative to today, when hype will peak, and we will see further accelerated investment in AI.

The Future Is Bright

There is enormous potential for this new wave of tech to impact humanity. For example, Google’s MedPaLM2 model outperforms human physicians to such a strong degree that having medical experts RLHF the model makes it worse (!).

Given the strong potential it will be exciting to see all the immense innovation in education, healthcare, enterprise and consumer software, and other aspects of life coming via this tech breakthrough.

We are only 8-9 months since chatGPT woke the world to this new era of AI, and exciting times are ahead as we follow this tech discontinuity down our timeline. It is the very earliest days of AI, and both peak hype and peak impact are still in the future. Lots more is still to come.

MY BOOK

You can order the High Growth Handbook here. Or read it online for free.

OTHER POSTS

Firesides & Podcasts

Markets:

Startup life

Co-Founders

Raising Money

Old Crypto Stuff: