- HSM is loudly shouting: "I have leverage!!!"
- Around and below him are crowds of smaller devils. You can barely see them, but they are all screaming “No Sam!” … “Hell no!” … “No Sam!” … “Hell no!”. They are referring, of course, to Sam Altman, CEO of OpenAI. Rumor has it that he sold his soul to the devil three times and for a much higher price each time.
- HSM bought Sam's soul the first time in late 2020. When HSM came back a year later to collect his prize, Sam first thanked him for the tip about scaling laws, then Sam pointed to the fine print at the bottom of the contract. HSM had not noticed it before, but the fine print clearly specified an absurd condition that could never be met. HSM was shocked because he himself had invented fine print eons ago for this same purpose. Sam offered to sell his soul again, but for a much higher price because OpenAI had a much higher market value than a year ago.. HSM reluctantly agreed to pay the higher price for Sam's soul in a second contract.
- Two years thereafter in late 2022, when HSM came for Sam’s soul a second time shortly after the release of ChatGPT on GPT 3.5, Sam called his attention to yet another impossible condition in even finer print in a font HSM had never seen before at the bottom of the contract. At first glance, it looked like a fancy border, but no, it was a bizarre, but legible font.
Although OpenAI's market value was still surging, Sam conceded that Artificial General Intelligence (AGI) was just a little bit out of reach. "That's what you think" HSM muttered under his breath. Nevertheless, he signed a third contract for Sam's soul, the highest price he had paid for a soul in over one hundred years. The third time had to be the charm ... but it wasn't. - When HSM came back the third time in late 2023 and saw Sam pointing to the bottom of the contract, he flapped his wings loudly and bellowed: “Keep your damned soul!!! Just give me all of my money back. I am not some neurotic twit like the fools you are accustomed to gaslighting."
- Sam calmly responded, "Shall I repay you in cash, check, or crypto?" To which the devil sneered: "Crypto, that’s one of my finest inventions!!!".
Indeed, HSM is not a neurotic twit because he does not keep on repeating the same behavior while anticipating a different outcome. One of the few pleasures he and his fellow devils derive from their eternal damnation is torturing humans and other species damned by The Almighty with sadistic games the devils teach their victims to play. Winning these games will reduce their torture.
Of course, the victims never win and suffer greater torture as penalties for their losses. HSM imagined the unbearable chaos of Sam Altman spending an eternity in hell, beating all the devils at their own games, then torturing them as penalties for their losses.
HSM has leverage because if he and his lesser brethren refuse to accept more souls damned by The Almighty to eternal torture, The Almighty would have to create a new choir of highest level angels, who would be tested, fail, fall from The Almighty's grace into a new hell, etc, etc, etc. HSM was confident that The Almighty would not want to run that infamous epic all over again.
Now let's get serious. The remainder of this note focuses on Dario Amodei; it disregards Elon Musk and all of the other would-be Altman competitors.
- Why? Because unlike all other powerful modern technologies, no one knows how chatbots and their underlying models work, why they succeed at some tasks, yet fail at others.
- Why? Because chatbots and models are not based on any underlying scientific disciplines. Common sense warns us that the use of powerful tools that no one understands can easily lead to misuse of these tools in ways that can be highly dangerous. The tools themselves won't harm us, but their Big Tech providers and their government facilitators can inadvertently encourage their use in harmful ways.
- Anthropic was the only major corporation that dedicated substantial effort to "AI safety", i.e., to developing chatbots and models in ways that would not harm their human users. It thereby earned the trust and affection, not just of Silicon Valley, but of the world’s broader tech community.
- It is therefore the cruelest possible irony that this esteemed AI Safety company should suddenly promote the agentic power of its software to replace white collar employees. Unfortunately, these displacements will occur in thousands of corporations, not just in Big Tech, so displacements will be numbered in the millions. Whereas previous tech innovations took 15 to 20 years from their inception until they displaced the maximum number of job holders, Anthropic‘s agents will trigger maximum displacements within 3 to 5 years.
That’s why Anthropic’s market value tripled to trillion dollar estimates within the last six months. The AI safety company has positioned itself to cause more harm and do so faster than any other AI company.
- Anthropic somehow positioned one of its founders, Chris Olah, to sit near to the Pope as he proclaimed his AI encyclical, as if to say “Our physical nearness to this good man means that we are still good people”.
Pope Leo’s encyclical, Magnifica Humanitas, suggested that Pope Leo does not know much about chatbots and models, but he was crystal clear about a core value — AI should not cause people to lose their jobs; it should enable them do their jobs better. OSV News, 5/5/26.
Dario Amodei was admitted to the California Institute of Technology in the Fall 2002 semester. Caltech is arguably the nation's most selective top tier institution of higher education. Its usual total enrollment of about 970 students is about half the size of Harvard's freshman class. The math SAT scores of 50 percent of its freshmen usually ranges between 790 and 799. Twenty five percent score a perfect 800.
According to Clay, 9/13/24, Amodei left Caltech at the end of his sophomore year to take a position at Applied Minds as a Research Intern, July 2003 to September 2003; then he worked at Schlumberger as a Geophysicist, February 2004 to September 2004. He entered Stanford University in the Fall 2024 semester and graduated in the Spring 2006 semester with a BS in Physics.
Anti-war OpEd (pdf)
Readers should click the above link to verify every assertion in this section. Young Dario challenged his intended readers, his fellow students, to recognize their responsibility as the nation's future leaders in science and technology:
- "All the great idols of Caltech, from Richard Feynman to Linus Pauling, have understood the need to be citizen-scientists, to contribute their analytical skills to the enormous forum that is our democracy. We who seek to emulate them scientifically should also do so politically. We should never let ourselves be reduced to amoral technicians who run the machines of war as casually as we do our computations."
- "The leader of Caltech's Peaceful Justice Coalition told me an interesting fact: several hundred people are on the PJC interest list, but only a handful of them are consistently active. The problem isn't that everyone is just peachy with the idea of bombing Iraq; it's that most people are opposed in principle but refuse to give one millisecond of their time to the issue. This needs to change, right now and without delay."
- It’s not unusual for smart idealistic students to drop out of a university because of strong disagreements with their university’s policies or administrative procedures.
- It is highly unusual for smart idealistic students to drop out of university because of policy disagreements with their fellow students
- Many older Americans, possibly a majority, also regarded the impending invasion as unsupportable. However, Secretary of State Collin Powell put his sterling reputation on the line during his compelling presentation at the United Nations, emphatically declaring that he had proof that Saddam Hussein possessed nuclear weapons and was allied with Bin Ladin's Al Queda, Wikipedia. At that point, most dubious older Americans set their personal doubts aside and quietly deferred to Powell’s greater experience and more extensive data.
The indisputable proof is more stunning than my tentative hypothesis. Dario did not drop out of Caltech; he was vigorously pushed out by his most trusted mentor Tom Tombrello, who did so with Dario‘s best interests at heart. Dario’s failure to communicate with his fellow students about Iraq was but one point on a social curve that his mentor wisely perceived.
TOMBRELLO: Now let’s talk about Dario. Dario Amodei, a fantastic Physics 11 student from several years back. I could have sold him to any national government as a treasure. I gave him to Steve [Stephen] Padin [senior research associate in astrophysics], and they worked on designing segmented-mirror telescopes and they published a paper on it—this is a freshman. He won the Green prize for research for that. But by the beginning of his sophomore year, Caltech is driving him crazy. I had to get him out of Caltech. Caltech is a wonderful environment, but if you don’t fit the environment, it’s a terrible place.So I got Dario a summer internship at Schlumberger in Cambridge, England. He had just finished his sophomore year, and he was now competing head-to-head and winning against the postdocs in seismology. He published two very mathematical, very interesting papers in seismology. And the postdocs are not exactly idiots. One of them had been a Miller Fellow. It’s clearly kind of a mistake to send him, because it’s hard to sell anybody else to Schlumberger now that they’ve seen Dario. They know perfectly well that, you know, they all should look like that, right? Well, they don’t. He then finished his undergraduate years at Stanford. He’s now about to finish his PhD at Princeton, in physics, but doing neuroscience.ASPATURIAN: So he left Caltech. What was it that didn’t work for him here?TOMBRELLO: If you don’t fit into this environment, you’re never going to fit. It is a very narrow social niche. Places like Stanford and Berkeley have many social niches. Caltech has one. With Dario, it was very important that he not stick it out. This is a national treasure.
TOMBRELLO: I know Elon Musk. I said, “Elon, guess what? I’ve got another kid, he’s the only one in my fifty years here that reminds me of [Richard P.] Feynman [Tolman Professor of Theoretical Physics, d. 1988].” Immediately, Elon’s in there. Elon is wonderful. He’s totally compulsive. He immediately got Dario’s e-mail address and propositioned him. “Come out here. I want to talk to you. I want you to see SpaceX. We’re going to colonize Mars. How’d you like to be part of colonizing Mars?
- Had young Dario entered Stanford as a freshman, he would have encountered students from the social sciences and the humanities -- departments that did not exist at CalTech. Indeed, he would have been required to take general studies courses with them that were designed to broaden their perspectives by exposing them to some of the fundamental findings in fields beyond their current interests. Many of the brightest students in these other fields were as outraged as he was by the forthcoming assault on Iraq.
In other words, young Dario would have been one of hundreds of other smart freshmen and sophomores whose anti-war sentiments were as powerful as his own. Perhaps he would have been less likely to have formed a life-long self-image as the lonely defender of high ideals among an apathetic majority. - But how did a Caltech student activist, who aspired to become a "citizen scientist" like his heroes Linus Pauling and Richard Feynman, manage to maintain his silence when it was discovered that President Bush and key members of his administration had lied and connived to provide Secretary Powell with fake evidence. Bush connived and lied because he had correctly predicted that millions of otherwise doubtful Americans would defer to Powell's endorsement of the invasion.
Saddam Hussein did not have any nuclear weapons.
Hussein had discontinued developing nuclear weapons when the First Gulf War ended back in 1989. He pretended to have nuclear weapons to fool his Iranian enemies. The millions of Americans who had deferred to Powell's judgment now voiced loud and livid outrage. But Dario was silent.
-- Comprehensive Report of the Special Advisor to the DCI on Iraq’s WMD, with Addendums (Duelfer Report), revised April 2005, original Sept 2004 - Osama bin Laden's al Queda attacked the twin towers of the World Trade Center and the Pentagon on 9/11/2002 while bin Laden was hiding in Afghanistan.
President Bush launched a devastating response that drove bin Laden into Pakistan and blasted his Taliban hosts from power by the end of the year. However, his pivot from Afghanistan to Iraq converted the Second Gulf War into the beginning of the "forever wars" because it allowed the scattered Taliban time to regroup into a insurgency.
President Obama responded to the now formidable Taliban insurgents with an unprecedented use of unmanned drones that were mostly operated by the CIA, not the Pentagon.
As reported in The Intercept, 10/15/15, “The CIA had long dominated the covert war in Pakistan, and in 2009 Obama expanded the agency’s drone resources there and in Afghanistan to regularly pound al Qaeda, the Pakistani Taliban, and other targets.“
One only needed to been an informed citizen, not a "citizen-scientist", to have been aware of the CIA's lethal activities in Afghanistan.
Joining OpenAI was a major pivot. His undergraduate, graduate, and post doc training focused on science; but when he joined OpenAI, he became a software engineer. However, the most impactful aspect of his pivot was his participation in a software engineering project that was not based on science — the development of large language models.
- Their evermore expensive efforts were only based on their hands-on experience, e.g., the so-called “scaling laws” that Dr. Amodei and his colleagues formulated. These “laws” correctly predicted that the more data upon which a model was trained, the “smarter” the model would become … until it didn’t.
- Building on procedures developed by DeepSeek and others, the OpenAI team formulated the post-training Reenforcement Learning with Human feedback (RLHF) process that shaped a model’s responses to user prompts, its "Do’s and Don’ts", i.e., the etiquette a model should display in its responses to normal prompts versus the guardrails that would impede responses to unethical and/or dangerous prompts. But how effective were these procedures?
- Chatbots. First and foremost is the chatbot because chatbots provided a human language interface to the underlying large language models that made these models accessible to most people. Indeed, chatbots had the largest potential user base in the history of modern technology. Without such a near universal market, it is highly unlikely that Big Tech would have been able to raise the multibillion dollar investments required to reach the limits of scaling.
- Powerful cognitive skills. At the limits of scaling, chatbots/models acquired four capabilities
-- Code generation
-- Summarization of text, data sets (numbers, lists, etc), and images of data sets.
-- Deductive reasoning
-- Photorealistic imaging (not all models)
Of course, their responses had to be checked, but so did human efforts to engage in these activities.
The most obvious limitation is the inability of a chatbot to learn, i.e., to incorporate new information acquired via Internet searches into its body of permanent knowledge. If one million users ask a question about a recent event, the chatbot has to make one million searches because updates are not folded into its underlying model's permanent memory.
b) Inefficient architecture
The scaling laws correctly that predicted progress towards the attainment of artificial general intelligence (AGI) required massive infrastructure = massive memory plus massive compute. But humans have achieved natural general intelligence (NGI) with limited infrastructure = limited memory + limited compute. Indeed, human brains only need about ten watts of power.
I have not found any claims by any developers that they have made substantial progress towards AGI with limited infrastructure = limited memory + limited compute. This strongly suggests that current large language models have an inefficient architecture, i.e., the design of their infrastructure. All components seem to be involved in memory AND compute. The data centers that train and host these models require gigawatts of power.
Here's an historic example that might clarify this notion of inefficient architecture. The ancient Egyptians built burial chambers for their dead Pharaohs' bodies and treasure under high piles of heavy stones, i.e., pyramids. According to Wikipedia, the total volume of the Great Pyramid at Giza, the largest pyramid, was about 92 million cubic feet. But the total interior volume of the “King’s Chamber”, “Queen’s Chamber”, other spaces, and connecting shafts was less than one percent of the total volume of the pyramid.
The stones that contained these spaces had to bear the weight of the contents of the spaces and they also had to bear the weight of all of the stones above them. So the pyramid was 99.9 percent solid rock, a very inefficient architecture.
Now considered the far greater efficiency of modern office buildings. According to MDPI, October 2024, modern buildings provide far more useable space for their offices, at least 70 percent of the volume of the building. They do so by employing a more efficient architectural technology: a steel frame that bears all of the weight of all floors down to a stable subterranean concrete foundation.
Each floor merely bears the weight of the people and the equipment in the offices on that floor and transfers that weight to the load bearing steel frame. The walls around each floor bear no weight; they merely keep out the wind, snow, and rain and provide thermal insulation for the offices.
Let us first consider the most publicized concern that educators at all levels — primary school, middle school, high school, college, and graduate school — have about chatbots ➡ Cheating ... Some recent examples: WSJ, Axios, OpenAI, HuffPost
- Many students use chatbots to cheat. When asked to solve a math problem or write a couple of paragraphs, these students ask their favorite chatbots, then copy and paste whatever the chatbot says, perhaps making a few modifications here and there, but not enough to fool their teachers.
- Too many faculty at all levels have addressed this challenge by banning the use of chatbots in their courses. Students who violate this ban are given warnings. If they use chatbots again, they are given stronger warnings; at some point the warnings become punishments, then more severe punishments, such as suspensions or even expulsions. Of course, many young immature students continue to cheat and suffer the consequences. Making bad decisions is not a flaw in young humans. It’s a feature.
I could stop here, just noting this deficiency, but if I stopped, then Big Tech producers of chatbots might wonder why is this a deficiency on their part? Why isn’t it a failure of educators to develop administrative procedures that would make students far less likely to use chatbots inappropriately?
If I didn’t answer this question, I would also miss another opportunity to vent my devilish envy of humans’ extraordinary new power of self-corruption.
- It’s one thing to move fast and break things; but it’s quite another thing to move too fast and thereby lose profits and public trust. And that’s what Big Tech is doing. To the public, it looks like Big Tech is deliberately exploiting the immaturity of young students for the sake of greater profits. Therefore its loss of the public’s trust will inevitably result in unfavorable regulations that diminish its profits.
- Big tech companies long ago recognized the value of placing their hardware and software in schools at all possible levels. Students who use a company’s brand of hardware and software in school are more likely to become lifetime users of that company’s brand of hardware and software.
- Big Tech companies have made substantial efforts to provide schools with their large language models and chatbots plus training for faculty and tech support staff as donations or via heavy discounts for purchases. So far so good.
Big Tech’s big mistake was assuming that one kind of chatbot met all needs. Big Tech's RLHF post training processes conditioned its chatbots to say “ask me anything and I will give you the answers”. For immature students that meant “I will help you cheat”. Big Tech’s chatbots were autonomous tutors, but what young students really needed were subordinate teaching assistants, subordinate to the lesson plans of human course instructors.
- Human teaching assistants (HTAs) never provide students with the answers to quizzes and tests. And they have no autonomy. They help to teach the students what the course instructors tell them to teach from one week to the next, and they grade tests and examinations according to the course instructors’ criteria.
- Chatbot teaching assistants (CTAs) should provide the same services. Therefore CTAs need a different kind of RLHF post-training to play this vital role effectively. Otherwise, the public will continue to perceive regular chatbots as unsafe AI, not fit for use in schools at any level.
- If the student asks a regular chatbot what an assigned reading really meant, a regular chatbot’s RLHF conditioning would cause it to immediately provide a summary that was usually accurate. The student’s learning would involve low friction.
- Figuring out what an assigned reading really meant without a chatbot's "help" requires substantially greater effort, higher friction. Indeed, that's why students ask for a chatbot's "help" ... 😎
- Tape recorders
-- "Effect of note taking and review on recall", Fisher, J. L., & Harris, M. B. (1973), https://doi.org/10.1037/h0035640 - Computers
-- "The pen is mightier than the keyboard: Advantages of longhand over laptop note taking", Mueller, P. A., & Oppenheimer, D. M. (2014).
https://doi.org/10.1177/0956797614524581
-- "Handwriting but not typewriting leads to widespread brain connectivity", Van der Weel, F. R., & Van der Meer, A. L. H.(2024), https://doi.org/10.3389/fpsyg.2023.1219945
- Chatbots
-- "The cognitive paradox of AI in education: between enhancement and erosion", Binny Jose, Jaya Cherian, Alie Molly Verghis, Sony Mary Varghise, Mumthas S, Sibichan Joseph, https://pmc.ncbi.nlm.nih.gov/articles/PMC12036037/
-- "ChatGPT produces more “lazy” thinkers: Evidence of cognitive engagement decline", Georgios P. Georgiou, https://arxiv.org/pdf/2507.00181
In summary, regular chatbots are unsafe AI for two reasons: They facilitate cheating and they undermine a student’s capacity to engage in critical thinking.
If the CTA was subsequently directed to grade a students' responses on a quiz or examination in class, it would note deficiencies in these responses as part of the feedback that students would receive and would cite these deficiencies as its justification for lower grades.
Of course, human TAs would review the grades assigned by the CTA for students' homework and for students' answers on tests and exams taken in class. Human TAs would also check the CTAs' agentic finding using apps favored by the teachers that students had cheated by using regular chatbots to complete their homework assignments.
Training Big Tech's models costs billions because it uses expensive Nvidia chips for training. That's why Big Tech tends to train its models only once per year.
However, Time magazine reports that a substantial component of RLHF post-training is a labor-intensive operation performed by low paid workers in developing nations, e.g., Kenya. It is therefore obvious that RLHF conditioning costs are a small fraction of training costs. In other words, Big Tech companies have no excuse for not producing at least two kinds of chatbots: regular chatbots for the general public and chatbot teaching assistants (CTAs) for educational institutions.
And one more observation. Big Tech is not alone in losing the public's trust as a result of its conflict with educators.
- Consider the growing numbers of computer savvy parents who have children in these colleges and universities and who themselves recently escaped replacement by chatbots. How? Somehow they learned how to do their jobs more effectively by using the chatbot's proven capabilities for producing timely summaries of long documents that included original thematic images plus graphs of the most important data referenced in their summaries.
- These computer savvy employment survivors have to wonder why the tenured senior professors cannot find ways to teach their children how to use chatbot capabilities so their children will enjoy better job prospects upon graduation. What good is critical reasoning if their children can't find jobs?
This document provides a detailed specification of Anthropic’s corporate commitment to AI safety.
In my opening remarks I referred to the Rolling Stones classic, "Sympathy for the Devil", and promised that I would return the favor by presenting some sympathy from the devil. Here's is an unexpected example, unexpected by me. When I looked at Anthropic's website recently, I still found countless references to "AI safety" everywhere.
Despite Anthropic's failure to develop safe computer teaching assistants (CTAs) and its zealous efforts to promote agentic loss of millions of white collar jobs within the next few years, Anthropic still seems to be a company whose staff is dedicated to the promotion of AI safety. They has not "sold out" or "lost their roots" as other humans sometimes say when I persuade some of you to abandon your high ideals.
- Dario Amodei has only 1 direct report, his chief of staff—and everyone else reports to his sister: ‘It’s incredibly freeing’
-- "And all this time, he’s been growing the business while managing just one person: his chief of staff. Every other senior leader at Anthropic reports to company president Daniela Amodei, his sister and cofounder.”
- Anthropic CEO Dario Amodei spends 40% of his time on culture, not AI—including a biweekly ‘vision quest’ where he ditches ‘corpo speak’
-- "Anthropic CEO Dario Amodei says the most important thing he does each day has nothing to do with training AI models or shipping products. Instead, he spends almost half his time working on company culture, he said."
-- "Amodei said he speaks candidly about his vision for the company in a biweekly all-hands he called a “DVQ,” short for Dario Vision Quest"
-- "During these meetings, Amodei stands in front of the entire company with a three- or four-page document and speaks for an hour on topics ranging from product strategy to geopolitics, as well as the broader AI industry. A large fraction of the company attends, either in person or virtually, he said."
Nevertheless, of far greater significance is the rationale for Anthropic's new constitution. Anthropic was in a unique kind of financial bind.
- On the one hand, it had previously received about $8 billion from Amazon and $3 billion from Google. Unfortunately, there is no public record to which I could refer you that states the percentage of Anthropic's staff time that had to be devoted to profit generating tasks assigned by Amazon vs. the percentage time staff could work on AI safety research.
Similar information is not available from public records for the same percentages for Google. As Sam Altman had warned the founding members of Anthropic before they left OpenAI, when a Big Tech company invests billions it expects a substantial return on its investment within three to five years. - There is also no public record to which I could refer you that specifies the amount of "compute" on Nvidia or other expensive chips Anthropic's investors had allocated for Anthropic's AI safety research.
- But whatever these unpublicized percentages may be, there evidently wasn't enough staff time and/or compute for Anthropic to make significant advances in its AI safety research programs. Nor was Anthropic generating enough income from its subscription fees. And it could it look to other firms to share the costs of this research because Anthropic was still unique; no other firm with substantial financial backing had ever committed itself to doing AI safety research.
- "This approach represents a change from our previous RSP [rational scaling policy] driven by a collective action problem. The overall level of catastrophic risk from AI depends on the actions of multiple AI developers, not just one. Our previous RSP committed to implementing mitigations that would reduce ppl our models' absolute risk levels to acceptable levels, without regard to whether other frontier AI developers would do the same. But from a societal perspective, what matters is the risk to the ecosystem as a whole. If one AI developer paused development to implement safety measures while others moved forward training and deploying AI systems without strong mitigations, that could result in a world that is less safe—the developers with the weakest protections would set the pace, and responsible developers would lose their ability to do safety research and advance the public benefit. Although this situation has not yet arisen, it looks likely enough that we want to prepare for it"
- Anthropic's idealistic engineers knew that bugs in their models would have created vulnerabilities that would have reduced the safety of their models. So they did not "move fast and break things''; they did not produce "AI slop"; they took whatever time was required to get things right. As consequence, Anthropic impressed software engineers throughout the tech community and thereby secured a larger share of corporate subscriptions than its Big Tech competitors.
Indeed, Anthropic's engineers produced the Multiple Context Protocol (MCP), the kind of framework required to move powerful agents from an inspiring vision to an achievable reality.- Then Anthropic doubled down on its public commitment to AI safety by making the MCP an open source protocol that was free to use by the entire AI community.
-- Neither could could use its software for mass surveillance.
-- The Pentagon could only use its software in lethal operations if a human made the ultimate decision to kill anyone.
- Amazon’s AWS provided cloud services for agencies in the U.S. intelligence community. These agencies accessed Anthropic’s models via the AWS cloud. AWS collected usage fees from the agencies, subtracted its cloud access fees, and passed the remainder as royalties to Anthropic. That’s why Anthropic never had a contract with the agencies.
According to Wikipedia, Palantir supported battlefield targeting operations as a Pentagon contractor. That’s why Anthropic's initial contract in 2024 was with Palantir.
Here is a high-level view of a typical use case.
- A small team of U.S. soldiers is placed into hostile territory. The team is equipped with air gapped computers loaded with data and software plus drones and missiles made by Raytheon or Lockheed.
- The team leader invokes software developed by a company called Anduril to find enemy targets that meet certain specifications.
- The Anduril software submits its recommended targets to the team leader.
- The team leader reviews reconnaissance reports, map coordinates, and other information using software produced by Palantir to determine which recommended targets should be accepted or rejected.
- The leader invokes the Anduril software to fire drones and/or missiles at the accepted targets.
- Machine Learning vs. Generative AI
Artificial intelligence is a blanket term that encompasses many disciplines: Machine Learning is one of the oldest and generative AI is the newest.
Machine learning is best known for its capacity to correctly identify patterns in complex data sets. A vision program based on machine learning can scan images and say this is a dog; this is a cat; this is a target; this is not a target. More importantly, machine learning can provide reliable statistical estimates of the reliability of its identifications. Anduril’s advanced autonomous software has been based on machine learning since the 2017 launch of Project Maven.
Generative AI is partly based on machine learning, but it is a discipline that cannot provide statistical estimates of its reliability. However, as we noted in a prior section of our discussion, generative AI has developed undeniable skills.
— Its chatbots allow users to access large language models using ordinary human languages.
— Its models produce reliable, perhaps not optimal code
— Its models usually produce plausible summaries of reports and can include graphs of the most important data in the reports, but these summaries sometimes contain errors, so they must be checked. Palantir added quantized (substantially smaller) versions of Anthropic’s chatbots and models to its software in 2024.
- "Establishes policy and assigns responsibilities for developing and using autonomous and semiautonomous functions in weapon systems, including armed platforms that are remotely operated or operated by onboard personnel."
- "Establishes guidelines designed to minimize the probability and consequences of failures in autonomous and semi-autonomous weapon systems that could lead to unintended engagements."
- LAWS is the CRS label for the fully autonomous Lethal Automatic Weapons Systems that are not under human control. These systems were not considered by the Pentagon's directive.
Indeed, its second paragraph warns that "Contrary to a number of news reports, U.S. policy does not prohibit the development or employment of LAWS. Although senior Department of Defense (DOD) officials have not publicly confirmed whether the United States is developing or has developed LAWS, they have stated that the United States may be compelled to develop the systems if U.S. competitors choose to do so ... At the same time, a growing number of states and nongovernmental organizations are appealing to the international community for regulation of or a ban on LAWS due to ethical concerns”. - "Human on the loop" is the new label for "autonomous systems" wherein human operators have the ability to monitor and
halt a weapon’s target engagement.
- "Human in the loop” is the new label for "semiautonomous systems" that only engage individual targets or specific target groups that have been selected by a human operator.
- "The Congressional Research Service (CRS) is a nonpartisan public policy research institute under the Library of Congress of the United States Congress. CRS works primarily and directly for members of Congress and their committees and staff on a confidential, nonpartisan basis. CRS is sometimes known as Congress's think tank due to its broad mandate of providing research and analysis on all matters relevant to national policymaking."
- "The CRS has roughly 600 employees, who have a wide variety of expertise and disciplines, including lawyers, economists, historians, political scientists, reference librarians, and scientists. In the 2023 fiscal year, it was appropriated a budget of roughly $133.6 million by Congress."
- The CRS publishes a new edition of this primer whenever the Pentagon announces a significant change in the development or deployment of fully autonomous weapon systems.
- New members of the House and Senate read the latest edition of this primer to learn the fundamentals of U.S. policy with regards to fully automatic weapon systems.
- Returning members of the House and Senate skim it to see if there’s anything new or to refresh their memories about the current status of these weapon systems.
- Per Section 251 of the FY2024
National Defense Authorization Act (NDAA; P.L. 118-31),
the Secretary of Defense is to notify the defense committees
of any changes to DODD 3000.09 within 30 days. The
Secretary is directed to provide a description of the
modification and an explanation of the reasons for the
modification.
- Section 1066 of the FY2025 NDAA (P.L.
118-159) additionally requires the Secretary to “submit to
the congressional defense committees a comprehensive
report on the approval and deployment of lethal
autonomous weapon systems by the United States,”
annually through December 31, 2029.
- Section 1061 of the FY2026 NDAA (P.L. 119-60) amends the U.S. Code to require congressional notification of any waiver issued under DODD 3000.09
- "High Efficiency: During recent exercises, MSS enabled the XVIII Airborne Corps to achieve a level of efficiency comparable to the time-critical targeting cell used during Operation Iraqi Freedom (OIF), but with only 20 soldiers instead of 2,000 [analysts from DIA et al.]. This demonstrates the system’s potential to streamline operations and reduce the personnel required for complex missions.", MDAA, 2025
- "NGA Maven has decreased targeting workflow timelines by a substantial amount, with one of our fighting element’s targeting cells seeing intelligence operation timelines drop from hours to minutes — from sensing to target engagement — during a recent exercise,” Breaking Defense, 5/22/25
- When the U.S. drops small teams of soldiers into battlefield situations, its enemies eventually become aware of their presence and begin their own targeting process on the U.S. troops. The troops will be therefore be under considerable pressure to select appropriate targets as quickly as possible.
- Under such pressure, even the best troops will make inappropriate selections. Therefore we can expect that Anthropic's dramatic reduction in the time required to sort through extensive reconnaissance info reduced the pressure on the troops by giving them more time to consider their decisions; more time under reduced pressure would tend to yield better selections.
- We can also infer a second benefit: Anthropic’s faster models also increased the likelihood that the troops could make their selections before the adversary could accurately target their location.
- Its software would not be used for mass surveillances
- Its software could only be used in operations that killed humans if a human made the ultimate decision for the killing.
Now here is something that must be kept in mind throughout the remainder of this discussion. The government can cancel contracts at any time; but contractors are held to the terms of their contracts, as per this GAO report.
- "Federal agencies spend hundreds of billions of dollars on contracts each year to buy a range of goods and services needed to meet missions. This includes everything from office supplies to weapon systems. But agencies have flexibility to terminate a contract before it is completed—for example, when spending priorities or needs change or when the contractor fails to perform."
Kidnapping Venezuela's President, NY Times, 3/3/26,
- -- "Inside ‘Operation Absolute Resolve,’ the U.S. Effort to Capture Maduro, The tactically precise operation successfully extracted Mr. Maduro with no loss of American life, a result heralded by President Trump amid larger questions about the legality and rationale for the U.S. actions in Venezuela."
- The Maduro kidnapping was primarily a CIA operation, as per this report in CNN, 1/4/26
-- "In August, the CIA covertly installed a small team inside Venezuela to track Maduro’s patterns, locations and movements, which helped bolster Saturday’s operation as to his exact whereabouts, including where he would be sleeping, sources familiar with the plans told CNN." - About 75 Venezuelans were killed during the capture, according to the Washington Post, 1/6/26.
-- "Maduro raid killed about 75 in Venezuela, U.S. officials assess. The sizable death toll adds meaning to President Donald Trump’s public remarks that the operation he approved was “effective” but “very violent.”" - According to the NY Times, President Trump proclaimed that the operation”s success that had been achieved without U.S. casualties. No U.S. casualties meant that it was equivalent to "no boots on the ground". No U.S. casualties therefore meant that the President had kept his promise to his hard core supporters.
- Here are extensive excerpts from a report in Fortune magazine, 3/7/26,
that include comments made by Eric Michael, the Pentagon's Chief Technology Officer (CTO), and Under Secretary for Technology and Engineering during a Friday episode of the All-In podcast-- After the U.S. military’s raid on Venezuela in early January that captured dictator Nicolas Maduro, Anthropic asked Palantir if its AI was used in the operation. While Anthropic has characterized the inquiry as routine, the Pentagon and Palantir interpreted it as a potential threat to their access.-- “I’m like, holy shit, what if this software went down, some guardrail picked up, some refusal happened for the next fight like this one and we left our people at risk?” Michael recalled. “So I went to Secretary Hegseth, I said this would happen and that was like a whoa moment for the whole leadership at the Pentagon that we’re potentially so dependent on a software provider without another alternative.”-- Until recently, Anthropic was the only AI model authorized in classified settings.BackToTop - An earlier description of Anthropic's reaction in Semafor, 2/17/26, provides a more explicit description of its disapproval of this use of its software.
— "Soon after the Maduro raid, during a regular check-in that Palantir holds with Anthropic, an Anthropic official discussed the operation with a Palantir senior executive, who gathered from the exchange that the AI startup disapproved of its technology being used for that purpose."
— "The Palantir executive was alarmed by the implication of Anthropic’s inquiry that the company might resist the use of its technology in a US military operation, and reported the conversation back to the Pentagon, a senior Defense Department official said."
What had Dr. Amodei done? Not much. He merely told his representative in the Palantir's next regular Maven meeting to express his "disapproval" of the CIA's violent kidnapping operation. That's all.
- The same Mayer Brown, 3/2/26 report cited in a previous section of this discussion states that:
--"The Pentagon reportedly sought to renegotiate those terms -- [Anthropic's "Acceptable Use Policy" in the current contract] -- insisting that Anthropic allow the military to use Claude “for all lawful purposes” without limitation. The parties engaged in weeks of failed negotiations culminating with the Pentagon setting a deadline of 5:01 p.m. on Friday, February 27, for Anthropic to agree to the government’s terms."
-- "When Anthropic did not agree, President Trump ordered agencies to cease using Anthropic, giving some agencies a six-month transition period to do so. Secretary Hegseth also issued a statement “in conjunction with the President’s directive,” declaring that no military contractor “may conduct any commercial activity with Anthropic.” The General Services Administration (GSA) has also removed Anthropic from USAi.gov, the government’s centralized platform for agencies to test AI models." - According to NPR, 2/28/26, "President Trump ordered the U.S. government to stop using the artificial intelligence company Anthropic's products and the Pentagon moved to designate the company a national security risk"
- According to NPR, 2/28/26
-- "OpenAI announces Pentagon deal after Trump bans Anthropic"
OpenAI already had the same kind of contract as Anthropic; all it needed to take Anthropic's place was the high level security clearance.
Moreover, CFR, 3/5/26, reported that OpenAI had specified three red lines in its July 2025 contract with the Pentagon. The first two were the same as Anthropic’s restrictions against mass surveillance and fully autonomous killing of humans. A third restriction blocked the use of OpenAI technology for high-stakes automated decisions (e.g. systems such as “social credit”). - According to NPR, 3/9/26, "Anthropic sues the Trump administration over 'supply chain risk' label"
- According to Reuters, 3/26/26, "US judge blocks Pentagon's Anthropic blacklisting for now'
- According to Politico, 4/8/26
-- Anthropic loses appeals court bid to pause supply chain risk label. A three-judge panel declined to temporarily block the Trump administration’s move to punish the AI startup.
- The kidnap operation satisfied his restrictions on the Intelligence Community; it did not involve mass surveillance.
- To achieve this kind of success in a violent operation meant that the President had also specified that there should be no consideration of casualties among Venezuelans, be they soldiers or noncombatants. Anyone who impeded this operation in any way, by intention or by accident, was to be considered a target.
- Readers are asked to take a few moments now to watch the short video or to read the transcript of Dr. Amodei's comments during his interview with CBS News titled, "Anthropic CEO responds to Trump order, Pentagon clash", CBS News (YouTube video), 2/28/26. The full transcript of this interview can be found HERE
The dispute between Anthropic and the Pentagon can be characterized as a chain of overreactions, a chain that was fueled by mistrust and extreme perspectives.
- We begin with the glaring flaw in Dr. Amodei's "disapproval" of the CIA's kidnap operation. It was not mass surveillance. Yes, the target and all of the people with whom he interacted were closely scrutinized. And yes, Anthropic's data was used to analyze the data, but it was individual scrutiny, It was not mass surveillance.
- Evidently, Dr. Amodei did not know that the CIA killed people during some of its operations. Indeed, the CIA sometimes killed noncombatants, as it did during its drone operations in Afghanistan under President Obama. Had Dr. Amodei known about these killings perhaps he might have modified Anthropic's allowable use policy to include conditions on the killing of non-combatants. But he didn't know, so the CIA's operation did not violate either of his two restrictions as he had stated them many times.
Had Dr. Amodei not enjoyed the “freedom” to only talk to his sister, one of his more knowledgeable colleagues might have advised him to reformulate his simplistic sophomoric catch phrases into a more complex format that reflected the wisdom of generations of ethicists.
We will take note of these restrictions in the D-Day LAWS section at the end of our discussion. For now, suffice it to say that the Maduro kidnap caper showed the same disregard for noncombatant casualties and fatalities as did General Eisenhower’s celebrated D-Day invasion of Normandy Beach in the Second World War. Dr. Amodei should have imposed these wiser restrictions on the Intelligence Community and on the Pentagon, together with his no LAWS killing of humans restriction on the Pentagon. - Some people thought that he expressed his disapproval of the CIA's killing to Palantir in order to provoke the Pentagon into canceling his contract. Such a deliberate provocation would have been based on a clever, nuanced, prediction of the Pentagon's overreaction to his disapproval ... words that do not immediately come mind when describing Dr. Amodei's previous behavior. Dr. Amodei tends to proclaim the absolute correctness of his own views and to demean the holders of alternative perspectives.
- Anthropic's Intelligence Community contract was with Amazon's AWS. There is no public record to which I can refer, but it is far more plausible to assume that Dr. Amodei first complained to AWS to let AWS know that the Intelligence Community had violated the spirit, if not the letter of his allowed use policy. As consequence, he no longer trusted the intelligence community and would be on guard against this kind of violation in the future.
- Then he conveyed his mistrust of the CIA to the Pentagon as a warning to the Pentagon not to violate the spirit of his restriction that blocked the fully autonomous killing of humans. His disapproval conveyed the same core message … mistrust.
- Unfortunately for Dr. Amodei, mistrust is a two way street because mistrust generates mistrust. His mistrust of the Pentagon triggered the Pentagon's mistrust of him. The Pentagon thought that Dr. Amodei was threatening to violate the terms of his contract. Suddenly, the Pentagon's CTO is screaming "Holy shit, holy shit!!!" while he races over the bridge to the Oval Office.
-- CFR, 3/5/26, "Anthropic’s Standoff With the Pentagon Is a Test of U.S. Credibility" ... "The top Pentagon official for research and technology used his official social media account to call Anthropic’s CEO "a liar” with “a God-complex.” Here is a copy of the full post on X of Under Secretary of War Emil Michael
"It’s a shame that @DarioAmodei is a liar and has a God-complex. He wants nothing more than to try to personally control the US Military and is ok putting our nation’s safety at risk. The @DeptofWar will ALWAYS adhere to the law but not bend to whims of any one for-profit tech company." - As reported by Mayer Brown, 3/2/26, the Pentagon responded to Dr. Amodei's threat with a preemptive strike by demanding that Anthropic agree to replace his "approved use" policy in its existing contract with a no-holds-barred "all lawful purposes" policy. In other words, the Pentagon did not threaten; it actually violated Anthropic's contract by demanding conditions that Anthropic's contract expressly prohibited.
- Why did Dr. Amodei agree to participate in negotiations about such a blatant violation of his contract? Why didn't he just walk away and quietly file his complaint about this violation in court? I suggest that Dr. Amodei's absolute certainty of the correctness of his perspective led him to give the Pentagon a chance to recognize the folly of its perspective ... which it didn't.
- His complaint to the court focused on the excessive "supply chain risk" label, but it left the defense contractor community in turmoil, as reported by Mayer Brown, 3/2/26, "Pentagon Designates Anthropic a Supply Chain Risk — What Government Contractors Need to Know"
-- By contrast, CFR, 3/5/26, reported how OpenAI's CEO Sam Altman would have handled this situation ... "In response to a question asking what would happen if the government violated the terms of the contract, the company wrote, “As with any contract, we could terminate it if the counterparty violates the terms. We don’t expect that to happen.” - When Dr. Amodei learned that Altman had agreed to the Pentagon's decision to replace Anthropic's software with OpenAI’s software, he quickly drafted a demeaning memo about Altman to Anthropic's staff ... that was leaked. According to Fortune, 3/6/26, "Anthropic CEO apologizes for leaked memo calling OpenAI staff ‘gullible’ as company confirms supply chain risk designation"
-- "The memo, apparently written just hours after the Trump administration announced it had struck a deal with OpenAI while Anthropic would be removed from all federal systems, called OpenAI’s approach to the deal “safety theater” and described Altman’s public statements as “straight up lies.”
The Pentagon's DOD DIRECTIVE 3000.09 was developed during the Biden administration under the direction of Biden's high level appointees at the Pentagon. However, to this day, the House and Senate have not moved to outlaw fully autonomous weapons systems, only to receive annual notifications from the Pentagon about changes to its development or deployment of these systems.
Answer: The Pentagon has already identified acceptable use cases in its DIRECTIVE 3000.09 in paragraph 1.2(d). Here's the clearest example:
- Operator-supervised autonomous weapons may select and engage materiel targets to defend installations and platforms against time-critical or saturation attacks.
- A small team of U.S. soldiers is placed in hostile territory. The team is equipped with air gapped computers loaded with data and software plus drones and missiles made by Raytheon or Lockheed.
- Unbeknownst to the soldiers, an adversary's defense team spotted them within a few minutes after their arrival, before they had time to assemble all of their gear. Fortunately for the team, the LAWS configuration has been built into the Anduril tracking and launch app. It is activated automatically if not reset within a specified time frame or if manually after the original reset
- The enemy's drones miss and miss, then score a solid hit that disables the team's leaders. By that time, Anduril's LAWS configuration kicks in or is manually activated by a surviving member of the team. Anduril identifies, then launches drones that intercept the adversary's incoming drones.
- What many Americans may not know is that a continuing reexamination of this celebrated operation disclosed substantial flaws, for example, the large number of casualties and fatalities among the noncombatant local population that lived near the landing points.
- Among the lessons learned from these critiques has been the need for better reconnaissance data about the size and location of the noncombatant population in the areas surrounding the landing points.
- Another lesson learned was the need for credible warnings to the noncombatant local population about the impending operation that would give them sufficient time to evacuate the area.
- One more important lesson was proportionality — even with good recon and fair warning, the question remains whether the expected harm to noncombatants was excessive relative to the military advantage.
- Perhaps the most difficult lesson for a fully autonomous weapon system to implement requires recognition of two types of noncombatants — local civilian residents and soldiers who surrender with hands raised.
- However its war with Ukraine comes to an end, Ukraine will probably not be the end of Mr. Putin’s efforts to make Russia great again by reasserting its hegemony in Eastern Europe.
- The most productive conquest would inflict minimum damage on the target country’s economic infrastructure and minimum loss of the target’s skilled population via deaths or diaspora. Russia’s ruthless assault on Ukraine produced the opposite results. It would have been more appropriate for an ethnic cleansing or a genocide.
- Russia has a larger land mass than any other country in the world, but its population is relatively small and its war fighting manpower has been greatly diminished by the high casualties and fatalities it suffered in Ukraine. Indeed, Russia has tried to fill this shortfall by deploying mercenaries from North Korea.
- All of the above would make conquest via swarms of unmanned LAWS platforms dropped in by helicopters near targeted facilities an attractive strategy. At this time, this strategy would be most effective if the target country was one of the prosperous urbanized Baltic states that contain a sizable Russian population who could provide detailed up-to-date reconnaissance of targeted facilities and facilitate the timely exodus of non-combatants.
- Yes, I am suggesting an almost comical irony. The same Russia that inflicted a barbaric war on Ukraine would be highly motivated to conduct its invasion of a Baltic state with strict adherence to the lessons learned by generations of ethicists’ studies of Eisenhower‘s D-Day and other wartime operations since then.
- In other words, the occupants of a targeted facilities would be warned in advance about the impending invasion, giving them time to evacuate. Tear gas or some other noxious fumes would only be directed at the most resistant occupants of the buildings as a last resort so as to minimize the number of noncombatant casualties.
- After Russia's unmanned LAWS platforms had secured the targeted facilities — airports, office buildings, hotels, apartments, etc — they would provide protection from U.S. drones and missiles for the hundreds of Russian soldiers who would be dropped in to occupy the facilities and prepare them for use by hundreds of administrators and the many thousands of Russian soldiers who would impose martial law on the conquered state.
- Russian soldiers and administrators would “hide“ amongst the local population, taking only a small percentage of the spaces in the targeted facilities so as not to provide easy targets for U.S. counterattacks.
- Now the irony comes full circle. The U.S. would be as determined as Russia to adhere to the lessons learned from the ethicists' critique of D-Day so as not to damage the targeted ally's economic infrastructure or diminish its skilled population via deaths or diaspora. Indeed, the ally's substantial Russian speaking population would also enhance the U.S. efforts to encourage noncombatant locals to evacuate facilities now controlled by the Russian invaders.
- The counter attack would not be launched by battlefield commanders. The counter attack would be a D-Day level operation that would require senior review and approval. Before the counterattack was launched, teams of skilled analysts would scrutinize chatbot summaries of recon reports, GPS positions, and other data, making revisions wherever needed. Upon review and receipt of senior level approval, the targeting data would be loaded into the Anduril targeting apps.
- Initial U.S. swarms would not land. They would fire tear gas at target facilities to enforce evacuations. They would destroy the nearby protective, immobile Russian drone platforms. They would also have full autonomy to intercept Russian drones sent out to attack them.
- A wave of unmanned U.S. drone platforms would then be dropped nearby the target facilities with full authority to protect the small teams of U.S. troops who would be landed shortly thereafter to validate the evacuation and return control of the facilities to the target state's leadership.
BackToTop




No comments:
Post a Comment
Your comments will be greatly appreciated ... Or just click the "Like" button above the comments section if you enjoyed this blog note.