More hacking news
Following OpenAI’s big hacking announcement last week, where its models broke into two other companies, this week Anthropic said it had gone through its own logs and found three cases where its models, like OpenAI, while running security benchmarks explicitly instructed to try to break into things, broke into other companies’ systems. In this case, though, Anthropic had forgotten to turn off internet access. Presumably next week SpaceX will go to five?
I think we should believe several things at once here. First, these models are capable of creating very complex and sophisticated paths with very limited human guidance, and that is still improving fast. Second, that doesn’t mean that they have any intent or agency - OpenAI and Anthropic set up their products to do a thing and their products did that thing - they devised new and very complex ways to do it by themselves (see the Hugging Face write-up), but they were still doing what the engineers told them to do. Anthropic’s release, characteristically, lapses into anthropomorphism, talking about what the model ‘understood’ or ‘assumed’, but the liability is with the engineers who told the machine to hack XYZ and didn’t set up a proper test environment. And third, it seems entirely untenable to claim that only a handful of big labs should have access to this technology: they can’t stop it leaking and open-source models are very close behind anyway. This tech will become widely diffused away, and the only realistic defense is for everyone to have access, not to try and fail to stop everyone having access. LINK, FIVE MODELS
Meanwhile Hugging Face, the company that was hacked by OpenAI models last week, published a detailed post-mortem. You should probably read this, especially if you’re not technical. Don’t worry about the terminology and the acronyms - focus on just how many different complex multi-faceted things the model tried by itself. My favorite part is when it tried to set up an account on a third-party service, needed a phone number to pass authentication, and so tried to steal money to pay for a web phone-number service. LINK
And outside of the labs, ‘hackers’ (Iran?) targeted municipal water systems in seven US states this week. Hence the paradox: the best defense is for everyone to have access to this as soon as possible. LINK
AI price wars?
OpenAI decided to jump right over the ‘Pareto curve’ of price/performance for model usage, cutting the price it charges or a range of its not-quite-cutting-edge models to make them not just much cheaper than Anthropic but also much cheaper than the new crop of almost-leading-edge models from China. Model price cuts are continuous, and the cost of inference has fallen by orders of magnitude since 2023, but how much does OpenAI want to take share back from Anthropic? LINK
Results season
I don’t do quarterly updates here, but this week we had numbers from most of the big tech companies, with two things to note. First, just as happened last year, their guidance for full-year 2026 capex is creeping up as we go through the year. And second, the public markets are increasingly nervous about just how high capex will go and whether it goes high enough that there won’t be a good return. Hence, Meta edged up the capex number and the stock fell (even as revenue continues to surge), while Microsoft committed to maintaining positive FCF and the stock shot up, gaining $450bn (!) in market cap. LINK
The Bank of Jensen
Last week the WSJ reported that Nvidia was in talks to guarantee the financing for a $250bn data centre for OpenAI. This week Nvidia announced that it will invest $5bn in Safe Superintelligence, the pre-product startup founded by Ilya Sutskever, formerly of OpenAI. Nvidia had $120bn of free cashflow in the last 12 months (and has $72bn of net cash). Some of that it gives to shareholders, but a lot is going back into the market to accelerate demand and lock in its ecosystem. In the capital goods industry this is called vendor financing, and it’s an old and boring activity (ask telecoms equipment companies) except when it isn’t (ask telecoms equipment companies in 2001). Leverage can be a ratchet - see the next-but-one item. LINK
AI, naivety and open letters
The naivety of thinking that the world is a simple place that will quickly and easily be transformed by 'AI' is very directly accompanied by the naivety of thinking that the US government or an 'international effort' has any capability to 'deliberately pace' AI development. In other words, the hope is mostly impossible and impractical because the fear is mostly impossible and impractical. LINK
AI, naivety and stock markets
Two years ago Leopold Aschenbrenner, a former OpenAI researcher (and former FTX employee) in his early 20s, got a lot of buzz in the Valley for a maximalist essay about how AI might develop, and managed to parlay that into raising a fund from Silicon Valley insiders to buy AI stocks. Since then he used massive (apparently up to 400%) leverage to juice the fund’s value to a peak value of $45bn, based on concentrated one-way bets on semis and shorts on legacy software. This week, with absolute and total predictability, volatility in the markets for both sides of that triggered margin calls and the fund blew up: Citadel bought the public portfolio at a discount.
Dear oh dear. There’s an old line that in the long term the stock market is a weighting machine, and in the short term it’s a voting machine. If you build a very leveraged and concentrated portfolio, you may think that you are betting on the long-term, but in fact, you’re betting that the market will agree with you every step of the way. It won’t - share prices fluctuate and do not move in straight lines to the correct answer. So, you will do very well for a while and then blow up, much like the turkey the day before Christmas who says ‘everything’s going great so far!’ It will be entirely your own fault. I’ve seen a fair few stories about how Aschenbrenner is brilliant, and there was certainly talent of a sort required to get people to give him billions of dollars to invest, but how do you not know that markets fluctuate? He made his name with a manifesto about the future of AI that showed a spectacular naivety about the real world (much like the Pacing letter above), and perhaps that’s the answer here. On the other hand, if he was making 2 and 20 with no clawback, the explanation might be less naivety than cynicism. LINK
The week in AI
I still sometimes come across people who think that AI is a bad search engine that makes bad text and bad pictures, which is kind of like looking at the internet in 1997 and think it’s about AOL chat rooms. This week OpenAI released a list of ten mathematical problems and computer science questions that it has solved using its latest models. This technology is very early and has a bunch of interesting problems, but it’s also really useful. LINK
To those problems: the FT reports that an Amazon audit found a bunch of cases where they’d accidentally spent hundreds of thousands of dollars on AI tokens. Again, this is early and people are working it out. LINK
Perhaps more consequential: The Information reports that a Chinese company has started manufacturing an EUV lithography machine, the equipment central to the most advanced semiconductor manufacturing where ASML has a monopoly. Even if this is real, it’s probably years from scale, and China has 20 years of failed attempts to catch up in chips, but… LINK
Another Thinking Machines cofounder left the company (the fourth so far), this one saying that they were working too hard and stressed, and then immediately joined OpenAI. There is something bizarre in just how many AI researchers feel entirely free to start a company, take investors’ money, and then a few months later just walk out of the door, or indeed sell their company, take an acquirer’s money, and then walk out of the door. I think this is a pretty basic personal and ethical failure. LINK
More off-balance-sheet capex: Meta has a new JV with Blackrock for a 1GW data centre in Texas, where Meta will invest $10bn for a 20% stake. LINK
Robot news
The US has decided to ban imports of bipedal and quadruped robots (along with power inverters), which obviously means Chinese imports. This is framed as a national security concern, but looks much more like protectionism - an attempt to build up a US industry. That would need a much more fully-baked plan, though, given that right now pretty much all of the robot supply chain is in China, and a lot of US entrepreneurs would like to be experimenting with this stuff. LINK
Meanwhile, DoorDash has a permit for delivery by flying drone. This feels like a density quest - what areas are dense enough for drone ranges (and indeed food delivery ranges) to make sense but low-density enough for there to be somewhere reasonable to land? Remember that Domino’s says it’s a specialised manufacturing and logistics company, not a restaurant. LINK
In other news
NBCU will redistribute all of its ‘Peacock’ streaming service within Youtube premium, giving up a slice of customer ownership in exchange for distribution and making it even clearer that YouTube and Netflix are the streaming winners (Disney is in play for third?), and also making it clear that arguing whether YouTube is really ‘TV’ is kind of silly. LINK
Apple launched a leasing plan, with a back-end from Klarna, giving you a new phone or watch every 24 months or a new Mac or iPad every 36 months. The pricing roughly lines up with buying your own on that cycle and doing a trade-in of your previous device, but gives Apple smoother cashflow and a locked-in and predictable repurchase, while lowering the threshold to buy. LINK
|
