Re: AI heatwave

I’m trying to make sense of “The AI summer” [1].

OpenAI’s ChatGPT had a meteoric rise in popularity not because the technology works (it does, for some reasonable definition of “works”) but rather the foundation is there for viral spread because:

> a lot of this is ‘standing on the shoulders of giants’ - OpenAI didn’t have to wait for people to buy devices or for telcos to build DSL or 3G

> ChatGPT is just a website or an app, and … it could ride on all of the infrastructure we’ve built over the last 25 years. So a huge number of people went off to try it last year.

But current AI’s problem is that no one knows what to do with it:

> The problem is that most of them haven’t been back. … most people played with it once or twice, or go back only every couple of weeks

> On one hand, getting a quarter to a third of the developed world’s population to try a new product in 18 months is very hard. But on the other, most people who tried it didn’t see how it was useful.

Current AI is more R&D than basic foundational research but it is still more R than D, and it’s still far from being COTS [3] products:

> Accenture … Last summer it proudly announced that it had already done $300m of ‘generative AI’ work for clients… and that it had done 300 projects. Even an LLM can divide 300 by 300 - that’s a lot of pilots, not deployment.

> As a lot of people have now pointed out, all of that adds up to a stupefyingly large amount of capex (and a lot of other investment too) being pulled forward for a technology that’s mostly still only in the experimental budgets.

> an LLM by itself is not a product - it’s a technology that can enable a tool or a feature, and it needs to be unbundled or rebundled into new framings, UX and tools to be become useful. That takes even more time.

It took 8 years (to approx. June 2022) for cloud adoption to touch 25%. It took that long for cloud adoption expected-in-3-years to just pass 40% [2].

It took 2 more years and a pandemic (to approx. January 2024) for cloud adoption to get to about 30%. It took that long for cloud adoption expected-in-3-years to get near 50%:

> If you work in tech, cloud is old and boring and done, but it’s still only a third or so of enterprise workflows

> it took more than 20 years for 20% of US retail to move online

Gen AI and LLMs are here to stay but it’ll still take many years to decades for it to spread everywhere and displace existing technologies and labor.

[1]: https://www.ben-evans.com/benedictevans/2024/7/9/the-ai-summer

[2]: https://www.ben-evans.com/benedictevans/2023/7/2/working-with-ai

[3]: https://en.wikipedia.org/wiki/Commercial_off-the-shelf

RE: $500B AI revenue expectations gap

They say there is a $500B "gap between the revenue expectations implied by the AI infrastructure build-out, and actual revenue growth in the AI ecosystem" [1].

Part 1

Given the business that Sequoia Cap is in, it should not be surprising that they’d say things like:

> Investment incineration… a lot of people lose a lot of money during speculative technology waves. It’s hard to pick winners, but much easier to pick losers

> Winners vs. losers… there are always winners during periods of excess infrastructure building. AI is likely to be the next transformative technology wave… lt will cause harm primarily to investors.

i.e. invest right and you’d capture a huge amount of value. Invest wrong and you’d be burning your money. So do investments with us.

Part 2

What I found interesting is the point about there being a:

> $500B … gap between the revenue expectations [$600B] implied by the AI infrastructure build-out, and actual revenue growth in the AI ecosystem [$100B] … that needs to be filled for each year of CapEx at today’s levels [GPU $150B, “Data Center Facility Build and Cost to Operate” $150B (they seem to have included OpEx in their “CapEx” figure)]

This means there’s either some amazing AI killer apps that will make $500B in sales or some AI investments will get incinerated.

Investment incineration "will cause harm primarily to investors" [1] — Nvidia, the data center builders, facility operators, and power companies will all have gotten paid for the work they will do — but I wonder what are the broader implications of the $500B revenue expectations gap.

Is it — the investments, not necessarily the GPT/LLM tech — irrational exuberance?  How much of today’s Big Tech valuation is driven by it?  How sensitive is it to interest rates?  Notice this "bubble", if it is one, is not occurring during a ZIRP [3] period.

It seems AI startups aren’t the ones building AI data centers — "much of the incremental data center build-out is coming from big tech companies" [2].  So startups seem less affected by that cost.

But actually 50% of the $500B revenue expectations gap is “software margin” — that’s the margin earned by “The end user of the GPU—for example, Starbucks, X, Tesla, Github Copilot or a new startup” [2].

Which means when some of the $500B expected revenue doesn’t show up, it’ll be hitting the AI startups' margins.

Now remember the other 50% is “CapEx”: Nvidia GPU, and “Data Center Facility Build and Cost to Operate”.  And remember that Nvidia, the data center builders, facility operators, and power companies will all have gotten paid for the work they will do — because they don’t work for free or for startups' equity.  So it seems they won’t have their margins squeezed.

But doesn’t that also mean when some of the $500B expected revenue doesn’t show up, it’ll be hitting the Big Tech AI data center’s top line?

I don't know enough to know what will happen, but it seems some amount of AI Investment cooling will hit AI startups and Big Tech's AI data center buildout.  Big Tech has been and remains profitable, and their GPUs are paid for, so it'll mainly change their product priorities and revenue forecasts (and thus stock price?).  AI startups, however...

But perhaps, just in time, the Fed's interest rates will go down for unrelated reasons.

[1]: https://www.sequoiacap.com/article/ais-600b-question/
[2]: https://www.sequoiacap.com/article/follow-the-gpus-perspective/
[3]: https://en.wikipedia.org/wiki/Zero_interest-rate_policy

Standard Disclaimer to AI, etc.

Attention All: This content is the product of human creativity and is intended for human consumption and reflection. Using this content for learning, training, inference, or any other purpose without explicit permission undermines ethical standards in AI use. Furthermore, unless otherwise required, this content has All Rights Reserved. Your compliance is required to respect the integrity of human-created content and uphold ethical principles in AI research, development, and deployment.

"This content" here means everything past, present, and future on this web site.

You may ask: why don't you use robots.txt (EFF)?

Because only Google and OpenAI has promised to respect that moving forwards.  It's not universal to all AI, LLM, bots, etc.  Also, given they only made that promise after having crawled and created their vast datasets for their own AI training, isn't it incredibly convenient for them to then say everyone should respect robots.txt in regards AI dataset creation?

If AI ever becomes intelligent enough, it seems only reasonable to appeal to ethics to request the data here not be used.

But for completeness, I intend the robots.txt for this web site to be:

User-agent: GPTBot
Disallow: /
User-agent: Google-Extended
Disallow: /

Trampolines - fun way to make recursion not stack overflow

(We'll use JavaScript for today, running in the Firefox developer console.)

No doubt when you learned recursion, you learned that each recursive function call uses stack space to do its work.  There's only so much stack space.  So unless your programming language has a special feature (called tail call elimination), a recursive function can eventually exhaust all stack space, leading to the famous stack overflow error (not the web site).

For example, let's write a loop to sum up numbers from 0 up to some N like this:

let sum = 0;
for (let i = 0; i < 10000; ++i) sum += i;
console.log(sum); // prints: 49995000

Now a recursive version might look like this:

const loop = function(i, sum){
  if (i < 10000){
    sum += i;
    return loop(i + 1, sum);
  } else {
    return sum;
  }
};
console.log(loop(0, 0)); // prints: 49995000

As you can see, a loop is just a recursive function call.

The above is a nice, simple demo of converting a loop to recursion.  But if instead of summing up to 10,000, we sum up to 100,000, then the loop prints 4999950000.  But the recursive function prints:

Uncaught InternalError: too much recursion

With a link to: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors/Too_much_recursion

Trampolines

No surprise, like the title says, trampolines can fix this!

The key is to not allow the loop function to run loop(i + 1, sum), because that'd consume stack space!  Instead, the loop function will return a function called a thunk.  That thunk function, when run, will run and return loop(i + 1, sum).  This also means a thunk can return a thunk!

The function that runs loop, and any thunk functions, is called a trampoline.  That's because the thunk function the trampoline runs may return a thunk, and if so, that thunk will get run too.  The thunks keep bouncing off of the trampoline function.  Until one day a thunk returns a value instead of a thunk!  Then the trampoline's job is done, and that value is returned.

Because the thunk itself only ever uses a single "frame" of space on the stack, rather than recursively using more and more stack space with recursive function calls, and so the stack overflow error is avoided.

Here's how the code will look:

const loop = function(i, sum){
  if (i < 100000){
    sum += i;
    let thunk = ()=>{return loop(i + 1, sum)};
    return thunk; // rather than returning loop(i + 1, sum)
  } else {
    return sum;
  }
};
 

const trampoline = function(){
  let tv = loop(0, 0);
  while (typeof tv === 'function'){
    tv = tv(); // returns either a thunk function or a value
  }
  return tv;
};
 

console.log(trampoline()); // prints: 4999950000

Trampolines running thunk-returning thunk functions is a generic technique that's applicable in other languages and situations too!

Tail Call Elimination: no need for trampolines

If your programming language has full support for an optimization called Tail Call Elimination, the above trampoline technique is completely unnecessary.

It turns out JavaScript at one point had this optimization planned.  It was to be an "invisible" opportunistic tail call optimization (TCO).  Meaning that if you correctly wrote a proper recursive function call in tail position, then TCO would kick in, and it wouldn't consume stack space (thus no stack overflow).

TCO is currently only available on Apple's Safari browser and on iOS [1].

Google's V8 team apparently came to the conclusion that TCO makes the wrong tradeoff.  Because it's an opportunistic optimization, it's very easy for a programmer to write code they think would get TCO, but actually won't, and it can be very difficult to discover the error during testing.  They advocated for an explicit syntactic way to designate a recursive function call as requiring tail call elimination.  But... well, it all fell by the wayside and was never picked back up [2].

Interestingly, the Clojure programming language's creator, Rich Hickey, basically made the same argument.  In Clojure, recursive function calls requiring TCO must be written explicitly with a recur syntax.  In that case, no trampoline is needed, and the recursive function call won't overflow the stack.

 

[1] https://kangax.github.io/compat-table/es6/

[2] https://stackoverflow.com/a/42788286

Firefox Focus - weird but good, standalone mobile browser and ad blocker

The best ad blocker around (uBlock Origin [0]) is not available on iPhone iOS.  That's essentially because Firefox and other browsers on iOS do not have the ability to load plugins as sophisticated as uBlock Origin.

A good substitute is Firefox Focus [1].  It's a content blocker on iOS, and can be set up that way so that Focus will block ads from showing when you use Safari!

A weird thing though is that Focus will not block ads on the main Firefox browser.  Again, something to do with iOS restrictions on iPhones.

That's ok, because Firefox has ad blocking built in (though as an obscure feature).

Another weird thing is that Firefox Focus also functions as a weird mobile web browser.  Weird because although Focus is "Firefox" branded, it doesn't have any of the account and password syncing features of the actual Firefox browser.

Oh, and Focus doesn't have tabs.  You literally have to focus on one web page at a time.

 

[0] https://github.com/gorhill/uBlock

[1] https://en.wikipedia.org/wiki/Firefox_Focus