Last update: Tuesday 5/28/24
No podcast this week
A. TL;DR summary of Top 4 stories
1. Copilot+PCs | 2. Build 2024 | 3. Anthro Peek | 4. Nvidia $$$
1) Microsoft announces Copilot Plus PCs with built-in AI hardware
Microsoft hosted two conferences this week. The first conference was on held on Monday. Up until then, Microsoft's efforts to monetize AI had focused on fees from its Azure cloud and subscriptions to Copilots for users of its Office and other cloud-based workplace applications. But on Monday Microsoft reminded the world that it also made PCs by introducing a new line of PCs from Microsoft, Dell, HP, and others that were far more powerful than any previous editions of Windows machines. These "Copilot Plus PCs" were more powerful because they were designed to host genAI models locally as well as invoke Microsoft's cloud based models.
The most important difference between these PCs and their predecessors was their inclusion of Qualcomm's "Snapdragon X Elite" NPUs (neural processing units), GPUs (graphics processing units), and CPUs (central processing units). Qualcomm was chosen because its chipsets were the only low energy consumers (required for laptops and other edge devices) that met Microsoft's AI-based demand that the NPUs be capable of running at a rate of at least 45 trillion operations per second (TOPS). However, Intel and Amd will be producing chips that meet these requirements later on this year.
A speed of 45 TOPS is fast, very fast. Compared to what? Compared to Apple's latest M4 chipset, whose NPUs max out at only 15 TOPS. Everyone expects that Apple's announcement of its full fledged entry into the genAI race at its June developers conference will feature powerful, privacy-oriented models that mostly run on their users' Macs and iPhones, rather than in the cloud. How will Apple's performance compare with the performance of genAI functions on Microsoft's more powerful Copilot Plus PCs? Suddenly Windows PCs are serious competitors for Macs again.
Readers are referred to the following concise explanation/discussion of the TOPs metric:
Readers are referred to the following concise explanation/discussion of the TOPs metric:
- "A guide to AI TOPS and NPU performance metrics", Peter Burns, Qualcomm, 4/24/2024
2) Microsoft's Build 2024
Microsoft's three day developers conference, Build 2024, was held in Seattle, WA, and online from Tuesday 5/21/2024 through Thursday 5/24/2024. CEO Nadalla's opening address on Tuesday sketched a broad framework for scores of innovations beginning with Microsoft's deepening partnership with Nvidia. But this TL;DR focuses on the announcement that was most directly related to the stunning reveal at Monday's meeting that Windows Copilot Plus PCs now had far more AI power than Macs.
Unfortunately, the editor must declare his deep disappointment that Build's announcement that Microsoft's new Phi-3 family of powerful small models would not be the drivers of AI functions on the powerful new Copilot Plus PCs. Instead, Microsoft was positioning Phi-3 as open source models that other developers could use. If Microsoft was seeking to persuade avid Mac users who were concerned about privacy and safety issues to trade in some (all?) of their Mac stations for the new PCs, then implementing AI services locally on powerful small models running on powerful desktop Macs or iPhones would have been be a doubly persuasive inducement.
Unfortunately, the editor must declare his deep disappointment that Build's announcement that Microsoft's new Phi-3 family of powerful small models would not be the drivers of AI functions on the powerful new Copilot Plus PCs. Instead, Microsoft was positioning Phi-3 as open source models that other developers could use. If Microsoft was seeking to persuade avid Mac users who were concerned about privacy and safety issues to trade in some (all?) of their Mac stations for the new PCs, then implementing AI services locally on powerful small models running on powerful desktop Macs or iPhones would have been be a doubly persuasive inducement.
Instead, Microsoft repeated its "platform" mantra that all of its AI services would be supported by cloud AND local (edge) components. No significant AI services would run only on cloud servers or only on the local machines.
Well, that still left the possibility that the powerful new Copilot Plus PCs would be more respectful of theirusers' privacy or less likely to produce unsafe responses. So the editor asked GPT-4o the following question. The chatbot's response can be found ➡ HERE
"Is there reason to believe that the high TOPS NPUs in Copilot Plus PCs are less likely to violate my privacy or produce unsafe responses?"
Giving the pathetic lack of progress that Microsoft and its partner OpenAI have made in the last 18 months in addressing safety and privacy issues, the editor of this blog found the chatbot's response to be extensive, intensive ... and vapid. Indeed, its response suggests that running Microsoft's AI software on faster, more powerful PCs will only make it more likely that the software will break bigger things faster ... :-(
3) Anthropic peeks into the GenAI black box
The most impressive capabilities of language models are the powerful new properties that emerge as the models get larger, i.e., capabilities that the models were not trained to exhibit. We might be content to take emergent properties as windfalls were it not for the fact that the models sometimes make errors, i.e., they make false assertions about real objects; and sometimes they hallucinate, i.e., they make statements about objects that don't exist.
Indeed, errors and hallucinations are the primary reasons why large enterprises are reluctant to make massive investments in generative AI services. Moreover, bias and safety are of great concern to government regulators. At8 the present time, gen-AI is not cost-effective, except for entertainment and/or casual use. Serious users must not only pay for the high cost of these services, whose high costs reflect the high costs of the expensive chips that run the services. Serious users must also pay the additional costs required to double check every assertion made by the genAI services ... and then check for safety and signs of bias. That's the bad news.
Now here's some good news from the leader of a research team at Anthropic, as reported by Wired:
Now here's some good news from the leader of a research team at Anthropic, as reported by Wired:
- "He leads an Anthropic team that has peeked inside that black box. Essentially, they are trying to reverse engineer large language models to understand why they come up with specific outputs—and, according to a paper released today, they have made significant progress.
Maybe you’ve seen neuroscience studies that interpret MRI scans to identify whether a human brain is entertaining thoughts of a plane, a teddy bear, or a clock tower. Similarly, Anthropic has plunged into the digital tangle of the neural net of its LLM, Claude, and pinpointed which combinations of its crude artificial neurons evoke specific concepts, or “features.” The company’s researchers have identified the combination of artificial neurons that signify features as disparate as burritos, semicolons in programming code, and—very much to the larger goal of the research—deadly biological weapons. Work like this has potentially huge implications for AI safety: If you can figure out where danger lurks inside an LLM, you are presumably better equipped to stop it.
The company’s researchers have identified the combination of artificial neurons that signify features as disparate as burritos, semicolons in programming code, and—very much to the larger goal of the research—deadly biological weapons. Work like this has potentially huge implications for AI safety: If you can figure out where danger lurks inside an LLM, you are presumably better equipped to stop it."
This is encouraging news. However, Anthropic's initial success will likely turn out to be the first few steps in ten thousand mile journey. Private corporations, no matter how well intended, are highly unlikely to support the subsequent ten thousand miles of required research. This kind of complex, expensive, long-term research process would be far more likely to succeed if it were managed by DARPA, a highly experienced and highly successful federal manager of expensive, complex tech development processes. See our previous blog note for a discussion of DARPA as the nation's genAI research manager
4) Nvidia's Earnings
As per the headline for the Forbes article, Nvidia reported a 628 percent increase in profits over the profits it earned in the same quarter in 2023. Nvidia also announced a whopping 10-For-1 stock split that enabled investors and employees to more affordably purchase whole shares of its stock. While these results exceeded everyone's expectations, they were not surprising given that Nvidia's earnings derived from its sales of the chips most favored by the the most successful providers of generative AI services that run trillion parameter large language models.
B. Top 4 stories in past week ...
- Microsoft
"Microsoft announces Copilot Plus PCs with built-in AI hardware", Max A. Cherney, The Verge, 5/20/24 ***
-- This story also covered by Reuters, Engadget, Ars Technica, Stratechery, Wall Street Journal, CNET (YouTube), NY Times, ... and Microsoft - Microsoft
"Microsoft Build 2024: news and announcements from the developer conference", Umar Shakir, The Verge, 5/22/24 ***
-- This story also covered by Mashable, Infoworld, CNET (video) ... and Microsoft
-- Stratechery's podcast interviews (transcript) (audio) with CEO Satya Nadella and CTO Kevin Scott provide insightful context for these announcements ... 5/23/24 - Hacks
"[Gen] AI Is a Black Box. Anthropic Figured Out a Way to Look Inside", Steven Levy, Wired, 5/22/24 ***
-- This story also covered by Fast Company, NY Times, Time, ... and Anthropic - Misc
"Nvidia Earnings: Stock Rallies As AI Giant Reports 600% Profit Explosion, 10-For-1 Stock Split", Derek Saul, Forbes, 5/23/24 ***
-- This story also covered by Axios, CNN, Reuters, NY Times ... and Nvidia
No comments:
Post a Comment
Your comments will be greatly appreciated ... Or just click the "Like" button above the comments section if you enjoyed this blog note.