Gen AI Costs

Introduction

2023 was the year of Generative AI (GenAI). Following the launch of ChatGPT in November 2022, OpenAI closed the biggest funding round of 2023 by securing a $10 billion multi-year deal with Microsoft. Anthropic and Inflection AI raised a combined $8.3 billion (Anthropic - $7 billion) taking the total investment in GenAI in 2023 to $25 billion.

‍

This number is expected to surpass $45 billion by the end of the year with$18.8 billion raised within the first 5 months already.

‍

The economic potential of GenAI could be in the trillions of dollars.McKinsey estimates it will add $7.9 trillion to the global economy. Goldman Sachs estimates it will add $7 trillion.

‍

OpenAI has a run rate of approximately $2 billion/year while Anthropichas been generating $2 million/month in revenue.

‍

Is this level of investment justified for GenAI startups? The optimism surrounding GenAI is clear. There is a lot of potential and a lot of hype.

Can GenAI make money?

Not unlike the early days of the internet, GenAI is currently more expensive for companies than it may seem. While the base version of ChatGPT is currently free, Microsoft's Github Copilot costs $10/month, and Anthropic's Claude 3 costs $20/month for individual users, these pricing models are more than likely to be loss leaders to get users used to the undeniable benefits of the technology.

While it is difficult to estimate costs associated with GenAI due to the lack of publicly available resources, the cost of running data centres can serve as a good proxy. NVIDIA is the largest provider of GPUs for data centres and as a result witnessed 2x growth in its revenue in 2023 compared to previous years - growth attributable to AI applications.

NVIDIA's data centres division generated $47 billion in FY 2024 which is a $32 billion increase compared to FY 2023. In a market where most cloud service providers buy GPUs from NVIDIA, $32 billion amortized over 10 years means that companies would be required to spend $3.2 billion/year on just chips from NVIDIA.

The $3.2 billion does not include other costs associated with running data centres. Rough estimates assume that total OpEx is nearly double this number ($6.4 billion). Assuming that cloud service providers will price their services at breakeven for end-AI tech providers, companies like OpenAI andChatGPT need at least $13 billion/year to make a 50% gross profit.

For instance, consider Anthropic's current enterprise level pricing for Claude AI. At $40/month, Anthropic will need 27 million enterprise seats/year to be profitable.

What does this mean for your company?

Consider Microsoft’s Github Copilot for this example. The majority of Github users are software engineers. They are likely to continuously experiment with and augment their code. Assume they use 100 prompts per day and each prompt costs Microsoft $0.01 (to remain consistent with reports that Microsoft loses $20/month per user from Copilot prompts).

Github currently costs $10/user per month for Copilot Individual and$39/user per month for Copilot Enterprise.

Key Risks:

-         Costs of company-wide deployment of GenAI tools (such as Github CoPilot) will have to increase 2x or 1.5x for Microsoft to make a profit

-         OpEx for end-AI providers as highlighted previously will remain high sothe price increases are expected to sustain over the next decade.

-         GenAI will result in productivity boosts but investment in GenAI will only be sustainable if the boosts increase revenue by 1.5x.

Can these productivity boosts be achieved?

GenAI has demonstrated tangible benefits. As it approaches its peak within the Gartner Hype Cycle, we will soon see disillusionment and realistic applications within the next decade.

‍

‍

In its current state, GenAI, much like the Dotcom era, is yet to demonstrate sustainable business models. GenAI represents a significant paradigm shift from what we thought was achievable through technology (also like the advent of the internet). EY has famously adopted wide GenAI deploymentand estimates that GenAI has resulted in a 15% to 20% increase in productivity across a range of tasks. Productivity boosts across some tasks, such as mass document analysis, is estimated to be around 70% to 80%.

While the benefits are clear, operating expenses (mainly associated with data centres) are burning through cash rapidly. As mentioned before, GenAI has been an attractive prospect due to its affordable pricing. Once users are locked in, tech providers will likely increase their pricing resulting insignificant expenses for enterprises.

Just like the initial excitement about the internet led to overpricing and then market crashes, the current enthusiasm around GenAI will inevitably lead to spectacular crashes and bankruptcies (like Babylon). Companies need to be careful to avoid making the same mistakes.

However, history also shows that high initial costs often lead to more sustainable ways of doing business. As technology improves and we start to seethe benefits of scale, we can expect the costs of running GenAI to go down significantly after the next 5-10 years. This will make it possible for more practical and profitable business models to emerge.

‍

The Dotcom Bubble

The introduction of the World Wide Web in 1989 generated a massive amount of hype for internet-focused start-ups. The NASDAQ Composite Index grew 5x between 1995 and 2000 as a result. The market was quickly disillusioned, andthe index fell from a peak of 5,048.62 in March 2000 to 1,139.90 in October 2002.

What led to such a dramatic collapse?

It is important to note that during the formative years of the internet, users thought that the realm of possibilities was boundless. The Dotcom Bubble is therefore characterized by speculative overvalued investments in tech start-ups following the analysis of metrics such as web traffic rather than viable business models.

Was it really boundless?

In the early days of the internet, users would need to wait several minutes to load simple web pages or were asked to pick up a book while they waited for an image to load. Commercialisation was difficult due to high operational costs and capital expenditure. The end-users paid high amounts of monthly internet fees to receive a fraction of the speeds available today.

Unsustainable business models and unjustified investments lead to the collapse of several dotcom companies.

‍

Conclusion

The landscape of GenAI resembles the Dotcom era, marked by immense potential and significant financial risks. The large investments in GenAI reflect optimism about its transformative capabilities, but also come with substantial operational costs, especially related to data centres and GPUs.

‍

Current low-cost or free access to GenAI tools is likely temporary, with price increases expected as user bases grow. Despite the high costs, early adopters report significant productivity gains, indicating GenAI’s potential value.

‍

As technology advances and economies of scale take effect, operational costs should decrease, leading to more sustainable practices and profitability. In the meantime, businesses must proceed with caution, investing in GenAI forits long-term potential rather than its current hype.

‍