Wellfed - Planet Jamie
⚛️ Atom RSS Feed
Home > Uncategorized > AI Skeptics: Liberal Democracy in the AI Era (with Daron Acemoglu)
August 17, 2026 Cathy O'Neil, mathbabe Leave a comment Go to comments
For this week’s AI Skeptics episode we talked to Daron Acemoglu, economist at MIT, about his new book, What Happened to Liberal Democracy?
Categories: Uncategorized
Homepage: https://biboumi.codeberg.page
Forge: https://codeberg.org/poezio/biboumi
Biboumi is an XMPP gateway that connects to IRC servers and translates between the two protocols. It can be used to access IRC channels using any XMPP client as if these channels were XMPP MUCs.
Ever found you’ve accidentally entered too many
git
s in your terminal and wondered if there’s a solution to it? I quite often type
git
then go away and come back, then type a full
git status
after it. This leads to a lovely (annoying) error out the box:
$ git git status
git: 'git' is not a git command. See 'git --help'.
What a git.
My initial thought was overriding the
git
binary in my
$PATH
and having it strip any leading arguments that match
git
, so we end up running just the
git status
at the end of the arguments. An easier way is to just use
git-config
’s
alias.*
functionality to expand the first argument being
git
to a shell command.
git config --global alias.git '!exec git'
Which adds the following git config to your
.gitconfig
file
And then you’ll find you can
git git
to your heart’s content
$ git sha
cc9c642663c0b63fba3964297c13ce9b61209313
$ git git sha
cc9c642663c0b63fba3964297c13ce9b61209313
$ git git git git git git git git git git git git git git git git git git git git git git git git git git sha
cc9c642663c0b63fba3964297c13ce9b61209313
(
git sha
is an alias for
git rev-parse HEAD
.)
See what other git alias’ I have in my
~/.gitconfig
, and laugh at all the typo corrections I have in there. (Yes, git provides autocorrection if you enable it, but I’m used to these typos working!)
Now
git
back to doing useful things!
More than half of the states in the US have joined together in an unprecedented lawsuit against Meta that goes to trial on Tuesday, accusing the parent company of Instagram and Facebook of deliberately designing addictive products that lured in young people and caused them harm.
The jury trial is taking place in federal court in Oakland, California , and is expected to last between six and eight weeks. The jury is expected to hear from Meta CEO Mark Zuckerberg , Instagram CEO Adam Mosseri and employee turned whistleblower Arturo Béjar .
“Meta designed a dangerous product for young users, knew it to be dangerous, and then lied to children, families and the community about how dangerous it was,” said Rob Bonta, California’s attorney general. The trial proceedings will be led by attorneys representing California , Colorado, Kentucky and New Jersey, though the suit was filed by all 29 states in what is known as multidistrict litigation.
The 233-page lawsuit, first filed in October 2023, alleges that Meta regularly collects data on children under the age of 13 without parental permission in violation of federal and state laws. The lawmakers claim in court documents that Meta “refuses to abandon its use of known harmful features” and that its motives are based solely on profit to “maximize its financial gains”.
The sweeping legal proceedings could have profound consequences for the social media company. The attorneys general say that if Meta is found liable, damages could be as high as $200bn – an amount equivalent to the company’s 2025 annual revenue. The lawmakers are also asking that Meta be compelled to change the design of its products to make them safer for children , which may have longer-term effects than a fine.
Meta denies all allegations and says : “Rather than sticking to the facts or the law, the states have instead decided to chase an outlandish payout.
“The State AGs may call this a landmark case, but their limited claims are unsubstantiated and their financial demands are vastly disproportionate,” the company said in a statement. “The AGs offer no proof anyone in their states was misled, claim benign features like having an additional Instagram account somehow harmed their residents, and attempt to penalize Meta for industry-wide challenges like age verification.”
The attorneys general also say Meta “developed and refined a set of psychologically manipulative” features designed to maximize people’s time on its apps. Those include an infinite scrolling recommendation algorithm, constant notification alerts, thumbs-up “likes”, and visual filters for altering one’s image.
The lawmakers say young people are especially vulnerable to falling prey to such features and that excessive time online can lead to increased depression, anxiety, eating disorders and other mental health issues.
Internal research conducted by Meta is expected to be presented at the trial, including, for instance, a survey of 2,500 teens conducted in 2019.
“Young people are acutely aware that Instagram can be bad for their mental health, yet are compelled to spend time on the app for fear of missing out on cultural and social trends,” the results read.
The federal trial comes just two weeks after a judge ordered Meta to pay $567m to New Mexico in a similar case brought by the state’s attorney general. This was the second court-ordered financial penalty for Meta in New Mexico, bringing the total it is responsible for paying the state to $942m. A state trial in Tennessee is now under way
after newsletter promotion
Families, school districts and other attorneys general have brought thousands of lawsuits against Meta and other social media companies in recent years. The plaintiffs hope a death-by-a-thousand-cuts legal strategy will induce Meta to change its social networks for the better.
In California, thousands of coordinated cases have been filed in state court against Meta, YouTube, TikTok and Snap. Meta and YouTube lost the first of those cases to go to trial in February, being ordered to pay $6m to the young woman who brought the suit. (TikTok and Snap settled before the case went to trial.) Two more lawsuits slated to go to trial this summer, one federal and one in California state court , also settled for undisclosed sums.
The lawsuits have borrowed from the legal approach used against tobacco companies in the 1990s, which focused on cigarettes’ addictive qualities and the makers’ knowledge that their products caused harm and resulted in a $200bn in 1998 and enforced changes to cigarettes’ marketing.
Russell Coleman, the attorney general of Kentucky, said he and his colleagues are using that strategy and aim to show the jury that “Meta concealed what it knew about the harm its products cause young people”.
“AGs are in the perfect position to get this done,” he added. “We did it with the tobacco settlement in the 1990s. We did it with the companies behind the opioid crisis. We’ll do it again with Meta.”
I grew up on the Internet. I spent a significant proportion of my childhood, and later my adult life, on message boards, blogs, social networks, chatrooms, and other online communities. I made friends, found jobs, and was exposed to so many new concepts thanks to the Internet. Many of my core beliefs were sharpened by ideas and discussions I followed online. It’s been a huge part of my life.
But today, the joy is largely gone. This wasn’t a sudden change: my experience with the Internet has been getting worse over many years. As it became increasingly monetized, thoughtful knowledge sharing and new ideas started being replaced by an ever-growing volume of attention-seeking, low-quality content. And today, largely thanks to LLMs, the ratio is so skewed that it doesn’t feel worth it.
I’m reminded of the video game Disco Elysium (spoilers ahead). In the game, the world is made up of distinct regions called isolas, separated by a nothingness called the pale. The pale is a gradually growing area that degrades information. Venture far enough into it and you’ll be left with a damaged mind. The pale covers most of the world, and some believe that it’ll continue to expand until it envelops everything. This sounds a lot like AI-generated content! On the Internet, we still have a few isolas here and there, but the pale is growing faster than we can handle.
I’ve occasionally managed to find isolas of interesting people on social media like Bluesky, but I’m increasingly skeptical of short-form content. Learning about the world through 500-character snippets, or 30-second videos, is not a serious way to engage with knowledge. It triggers our worst instincts in terms of tribalism and groupthink and makes us intellectually lazy. This is true of short-form content whether you’re experiencing it on Instagram, TikTok, X, Mastodon, or something else.
Longer-form content like blogs isn’t safe, either. I’ve stopped reading Hacker News because too many stories on the front page are clearly written by an LLM. The writing is bad, and while there might be an interesting idea in there, it’s equally likely to be empty filler content. The effort of writing an article used to indicate that the author felt they had something worth saying. That’s no longer true. If the author couldn’t be bothered to write it, why should I bother reading it?
The final nail in the coffin was work. It’s incredibly hard to get to know the people you’re working with when everyone is slinging LLM outputs to each other. And this isn’t some anti-AI screed — I’m all-in on agentic software engineering! My issue is with communication more broadly. When you have no sense of what your coworkers believe, or how they think, all the joy and connection of collaboration disappears. I hope that we’ll collectively decide that it’s rude to pass off LLM writing as your own, but I wouldn’t bet on it.
The incentives on the Internet have been skewed for a long time, but there were always isolas of lovely people and high-quality thinking. These isolas are now much harder to find in the pale of slop, and I don’t see a way back.
O ne of the stranger things about this dizzying, headlong moment in time is that it doesn’t have much of a past. Everything is about the future of this, the future of that: the future of work, the future of humanity, the future of the planet. It’s as if everyone is screaming (some ecstatically, most terror-stricken): robots are taking over the world! Meanwhile, you can’t put down your phone, unplug, delete your AI apps, tell Zoom to piss off; it feels as if you are racing toward something, and can’t stop, or look back, or think straight.
But of course this weird moment in history does have a past. It comes from somewhere. It wasn’t inevitable. Things could have turned out differently. They still can. And if you look back, one thing you’ll see is that a lot of people saw this all coming. They foresaw our present when it was still the future. They thought it might turn out fabulously, a computer utopia. And they worried it might turn out disastrously. Above all, they issued warning after warning about the rise of what I call the artificial state, the replacement of the liberal democratic nation-state, the consent of the governed, with the rule of people by machines. They warned of a future in which democratic deliberation would be replaced by prediction by calculation, the public sphere would be supplanted by data-driven commerce, and drones would take the place of the demos. Why didn’t anyone listen?
B etween the 1950s and the 1980s, computers got smaller, faster, cheaper and more powerful; computer languages became more sophisticated; and electronic data, once a molehill, became a mountain. As data processing and general-purpose computers moved from the military to industry, business and medicine, sizeable chunks of public life became automated, driven by computers, calculations, networks and algorithms that did the political work of rallying support, running campaigns, communicating with constituents and crafting policy, especially in the world’s wealthiest nations.
The adoption and diffusion of such technologies led to wild-eyed speculation about their possible social and political consequences, a discussion that, characteristically, ricocheted between dreams of a technological utopia and fears of an artificial state.
In 1981, for instance, the Japanese sociologist Yoneji Masuda, a theorist of what he called “information society”, predicted the arrival of either a computopia that would bring about the eradication of all distinctions of class and nation and “a symbiosis in which man and nature can live together in harmony”, or a compudystopia that he called an “automated state”: government by machine. Yet the two possible futures had much in common, since the path toward either had already led humans “to neglect the need for coexistence with nature, while our impact on nature has grown immeasurably”.
A turning point in the story of the automated state had come in 1958, when a team of American researchers and ad men built a machine that they claimed could both predict and influence voting behaviour. That year, a landmark conference on the “mechanisation of thought processes” took place at the National Physical Laboratory in Teddington, near London. Speakers included French, Israeli, Russian and Hungarian scientists, and they talked of a future in which computers would one day compose music, resolve legal disputes, control air traffic, conduct surgery and, in the administration of government, replace not only clerks but executives.
In New York, Edward L Greenfield, the head of a Madison Avenue advertising firm, drafted a proposal for what would become known as a People Machine. Greenfield proposed to compile hundreds of thousands of punch cards – election returns and public opinion surveys and census data – to build an “information bank” that could sort voters by types and their opinions by issues. “There is nothing mysterious about this on the surface, the input to the machine being the information about real individuals obtained in surveys”, Greenfield and the MIT political scientist Ithiel de Sola Pool wrote in 1958. But “once this information is inside the high speed storage facilities of the machine, it is a different world”. The machine would operate as a “macroscope” that could simulate the whole of the US electorate.
The next year, Greenfield and Pool and IBM’s Alex Bernstein formed a company called the Simulmatics Corporation. By “simulmatics”, they meant the automated simulation of human intelligence. They believed they had invented “the A-bomb of the social sciences”.
Their first client was the Democratic National Committee. In 1959, the DNC hired Simulmatics to run simulations on an artificial electorate and tell the party’s nominee what to say, to whom, and when. Not everyone was enthusiastic. “I shudder at the implication for public leadership of the notion … that a man shouldn’t say something until it is cleared with the machine,” wrote the historian Arthur Schlesinger.
After John F Kennedy won the nomination, his campaign hired Simulmatics to run predictions on an IBM 704. When Kennedy won, many in the press attributed the victory to Simulmatics’ People Machine. As the New York Herald Tribune put it, “a big, bulky monster called a ‘Simulmatics’” had been Kennedy’s “secret weapon”.
Simulmatics went on to new campaigns, heralded by the press. But the People Machine turned out to have been ahead of its time. In 1970 Simulmatics went bankrupt, unable to find enough paying clients for its services. The problem wasn’t the computers, or the model. The problem was an insufficiency of data. When Bernstein met with book and magazine publishers and film distributors in hopes of selling a market simulation of book sales, magazine subscribers and ticket sales – the kind of microtargeting done decades later by companies such as Amazon and Netflix – he reported that there was no way to build a model for any of these industries.
Simulmatics’ only real successes were in politics, where the company could rely on large caches of data: censuses, election returns and public opinion surveys. But in this sector, the company’s work raised considerable alarm. It also inspired prophecies of political doom, including two dystopian novels. One was published in 1964: The 480, a political thriller in which a barely disguised “Simulations Enterprises” meddles with a US presidential election. It was written by Eugene Burdick, who’d been asked to join the company but had declined, for reasons that became clear in The 480, in which he described Simulmatics as part of a sinister political landscape.
The new underworld is made up of innocent and well-intentioned people who work with slide rules and calculating machines and computers which can retain an almost infinite number of bits of information as well as sort, categorise, and reproduce this information at the press of a button. Most of these people are highly educated, many of them are PhDs, and none that I have met have malignant political designs on the American public. They may, however, radically reconstruct the American political system, build a new politics, and even modify revered and venerable American institutions – facts of which they are blissfully innocent.
Burdick died in 1965, but his concerns about an automated state were bubbling up all over the place. In 1966, in The Myth of the Machine, the American critic Lewis Mumford lamented the rise of “cybernetic intelligence”, warning that under its “automatic operation … man will become a passive, purposeless, machine-conditioned animal whose proper functions, as technicians now interpret man’s role, will either be fed into the machine or strictly limited and controlled for the benefit of depersonalised, collective organisations”.
Mumford described technological determinism – the idea that machines determine the course of history and represent a march of progress – as “a radical misinterpretation of the whole course of human development”. This mistaken belief, he wrote in The New Yorker the next year, had to be abandoned “if we are to get an adequate grip on our mechanised culture before we lose both our consciousness of human purpose and our confidence in being able to control our own creations”.
No one really ever paused to get that grip. Instead, data collection became crucial to industries that began accumulating, as stored computer files, everything from credit card transactions to car rentals to library checkout records, which made possible the kinds of predictions that Simulmatics Corp had only been able to dream of. With so much data, computers – increasingly linked together – stood poised to become far more than simple question-and-answer devices.
W hile all this was going on – the accumulation of data, refinements in computer architecture and the use of computers not only to calculate but also to communicate – the best thinkers of the age wondered what it might mean for humanity: computopia, or the automated state? The answer appeared to depend, in large part, on what these technologies might mean for humans’ capacity to understand the world. As ever, the nature of government would depend on the nature of knowledge. “Organised knowledge puts an immense amount of power in the hands of people who take the trouble to master it,” as a director of medicine and science at the Rockefeller Foundation put it.
Meanwhile, the nature and form of knowledge was changing. In 1965, the brilliant engineer JCR Licklider, the man most appropriately credited with inventing the internet, wrote Libraries of the Future, in which he considered the many disadvantages of books. “Surveying a million books on 10,000 shelves,” he said, is a nightmare. “When information is stored in books, there is no practical way to transfer the information from the store to the user without physically moving the book or the reader or both.” But convert books into data that can be read by a computer and you can move data from storage to the user, and to any number of users, much more easily.
Using the contents of all the books held in the Library of Congress as a proxy for the sum total of human knowledge, Licklider calculated its size in 1960 and estimated that it was doubling every year. Using these numbers, the sum total of human knowledge, as data, would be about a dozen petabytes in the year 2020. No one really knows how big the internet was in 2020, but some people said it was dozens of zettabytes. If they were right, Licklider was way off. In any event, the library of human knowledge seemed, suddenly, breathtakingly vast.
Would the libraries of the future be available to everyone? And what about the libraries of the present: the troves of data in the possession of governments? In the US, a proposal to establish a National Data Center, meant to provide, for electronic data, what the Library of Congress provided for books and the National Archives for manuscripts, was abandoned due to privacy concerns. (Critics dubbed the proposed National Data Center the “snooping machine”.)
Licklider proved an excellent futurist. In 1963, he’d sent a memo to colleagues at the Advanced Research Projects Agency (Arpa), proposing “to develop a capability for integrated network operation”, what would become first Arpanet and, eventually, the internet.
Licklider’s vantage on this project, and his involvement in most of the cutting-edge research done in the 1960s, gave him exceptional clarity about the future. In April 1968, in an essay for the magazine Science and Technology, Licklider and the psychologist Robert W Taylor predicted that “in a few years, men will be able to communicate more effectively through a machine than face to face”. They also predicted the emergence of “online interactive communities” that would change how people engage with each other: “Life will be happier for the online individual because the people with whom one interacts most strongly will be selected more by commonality of interests and goals than by accidents of proximity.”
Simulmatics Corporation’s co-founder Ithiel de Sola Pool also contributed an essay to that issue, in which he predicted that developments such as personalised newspapers would lead to an era of hyperindividualism. “A reader starts by asking for a digest of the news,” Pool imagined. “He can ask for details on any story that interests him.”
Pool believed the hyperpersonalisation of news would transform politics. “In the 21st century, the sort of critic who now attacks conformity in society may be complaining of an atomised society,” Pool predicted . “Modern technology, he’ll assert, has destroyed our common cultural base and has left us living in a little world of his own.”
He was not wrong. “In the coming atomised society, the information the citizen gets will arise from his own specific concerns,” he wrote, and “when everyone can select his own fund of information, the political problem of gathering an effective body of support behind a particular rational issue or candidate will become very different and very much more difficult”.
M ore common in the 1960s and 70s than Pool’s predictions of an atomised society under an automated state – or at least louder, and more attention-getting – were predictions of computer-driven socialist revolution or democratic liberation as types of computopias. In Chile between 1971 and 1973, during the rule of the democratically elected Salvador Allende, cyberneticists developed Project Cybersyn, a network whose chief purpose was to manage Chile’s socialist economy. Political reality intervened: the entire project was abandoned after Augusto Pinochet took power in a coup in 1973.
Elsewhere, optimism only grew. “Ready or not, computers are coming to the people,” Stewart Brand predicted in Rolling Stone in December 1972. The personal computer, Brand predicted, would bring “power to the people”. Brand, an LSD advocate, had studied at Stanford, and he described the arrival of Arpanet as the best news “since psychedelics”.
Computopianism found many adherents. Writing in 1975, the Israeli-American sociologist Amitai Etzioni and his co-authors predicted that the computer revolution would bring a return to “the form of democracy found in the ancient Greek city-state, the kibbutz and the New England town meeting, which gave every citizen the opportunity to directly participate in the political process”. Historian Daniel Boorstin, the librarian of Congress, predicted in 1978 that the new ties of communication would create new ties of community, a development he called the “republic of technology”.
Yet there were, as ever, sceptics. Joseph Weizenbaum , an MIT engineer who in 1966 had developed Eliza, the first of what would come to be called a chatbot, objected to the development of artificial intelligence as leading, inevitably, to the automation of the functions of the state. Writing in 1976, he described as “obscene” any project that proposed “to substitute a computer system for a human function that involves interpersonal respect, understanding and love”. He also warned against research that would one day result in “a fully automated battlefield”, and, as he wrote bitterly, months after the US evacuation from Saigon: “I see no reason to advise my students to lend their talents to that aim.”
Licklider was himself too careful a thinker to subscribe to either prediction: computopia or an automated state. In 1969, he predicted that by the year 2000 “an international network of digital computer communication networks” – he called it the multinet – would serve “as the main and essential medium of informational interaction for governments, institutions, corporations and individuals”, replacing everything from the telephone and the postal system to shops and banks.
Licklider was somewhat less confident about what the multinet would mean for politics. He knew that “interactive politics would function well only to the extent that the citizens were informed” and warned of the possibility of “new vistas for dirty tricks”, including “clandestine artificial intelligence programs, searching through the databases, altering files, fabricating records and erasing their own audit trails”. Nevertheless, Licklider echoed Brand when he concluded that “computer power to the people is essential to the realisation of a future in which most citizens are informed about, and interested and involved in, the process of government”. Yet society seemed to be moving in a different direction, towards an automated state rather than one in which citizens became more active participants.
after newsletter promotion
A s the 1970s came to an end, consultants employed by Ronald Reagan’s presidential campaign invented their own People Machine, a computerised political information system. A reporter explained that it worked like a massive chessboard on which the Reagan campaign could try out possible scenarios: “If the unions scare half their members about Reagan’s labour record, should he step up his attacks on Carter or try to rebut their specific claims? Or, if John Anderson’s vote begins to drop, should Reagan add a campaign stop in Connecticut, or can he afford to cancel one?”
In 1980, it helped carry Reagan to victory. The language of an “information society” yielded to the language of an “information revolution”. In 1982, Time named “the computer” man (machine) of the year. “The computer will smash the pyramid” of wealth and render all people equal, one giddy forecaster promised in 1984.
Yet the metaphor of revolution was ill-considered, the political theorist Langdon Winner argued, because revolutionaries have answers to questions such as whether their movement seeks “to uphold a valid ideal of human freedom” and “Does it aspire to a system of democratic rule?” and “Will there be frequent, open elections?” By contrast, “the computer revolution is conspicuously silent about its own ends”. And, as Winner saw it, little evidence supported any of these computopian predictions. So far, “current developments in the information age suggest an increase in power by those who already had a great deal of power, an enhanced centralisation of control by those already prepared for control, an augmentation of wealth by the already wealthy.” He was shouting into a void.
“Scarcely a new invention comes along that someone doesn’t proclaim it as the salvation of a free society,” Winner remarked. There seemed no better example than the 1984 television advert for Apple’s new Macintosh personal computer. “Today we celebrate the first glorious anniversary of the information purification … a garden of pure ideology,” intoned a thunderous voice, as masses of humans, hairless and looking like androids marched in lockstep, and sat in rows to watch Big Brother on a giant computer monitor. They were dressed in grey uniforms, as if they lived under fascism. “On January 24, Apple Computer will introduce Macintosh,” the ad continued. Apple was suggesting that only its personal computer could prevent the rise of a totalitarian state machine. “And you’ll see why 1984 won’t be like 1984.”
Y et there were already signs that networking computers could lead to or contribute to social decay. Usenet, an online bulletin board, opened in 1980 on what was sometimes called the “poor man’s Arpanet”, a dial-up network linking universities and research centres. It offered users the ability to join newsgroups, and it included a reply function. Usernames could be anonymous. “Together immateriality and anonymity were an invitation to test social limits and boundaries,” the University of Winnipeg scholar Jason Hannan wrote. “Liberated from the personal repercussions and consequences of life offline, some newsgroup users discovered a sordid pleasure in transgression for its own sake.”
Sometime in the 1980s, this practice came to be called “trolling”. Viral hate dates to the 1980s as well. The American neo-Nazi George Dietz opened a bookstore in West Virginia in 1974 and began publishing a small subscription magazine called Liberty Bell. In 1983 he announced that he’d been “working, for the past two weeks, until four to five o’clock in the morning, trying to learn ‘computerese’ so that Yours Truly may talk to that monster in ITS language and on ITS own terms.” He soon after launched the first white supremacy bulletin board, Liberty Bell Network, on an Apple IIe personal computer.
His efforts were quickly followed by the launch, in 1984, of more bulletin boards, including the Aryan Nations Liberty Net, Aryan Nations/Ku Klux Klan Computer Net and White Aryan Resistance. Members of these groups also brought their style and their ideas into discussion groups on America Online and CompuServe. As the internet became the world wide web, these efforts disappeared, replaced, in the 1990s, by websites and, a decade later, by social media platforms.
Newcomers tended to break the existing conventions of the very early internet era by trolling, by instigating and participating in flame wars (an early term for bitter online arguments), and by being, generally, obnoxious. Old hands grew alarmed. “When i went into cyberspace i went into it thinking that it was a place like any other place and that it would be a human interaction like any other human interaction,” wrote Carmen Hermosillo, known by the username “humdog” in 1994, but “i was wrong when i thought that … it is a black hole; it absorbs energy and personality and then re-presents it as spectacle.”
Alarmed cyberspace veterans tried to coach newcomers in “netiquette”, as in a set of guidelines written by Sally Hambridge of Intel in 1995. “A good rule of thumb: be conservative in what you send and liberal in what you receive. You should not send heated messages (we call these ‘flames’) even if you are provoked.” Another rule concerned email, online chatting and online messages: “Remember that the recipient is a human being whose culture, language and humour have different points of reference from your own. Remember that date formats, measurements and idioms may not travel well. Be especially careful with sarcasm.”
The hope that the internet would obey rules existed at one end of a spectrum; at the other end was the more widely held enthusiasm for the idea that the internet would be a place without rules. Acolytes of Brand saw it as a mystical counterculture utopia called Cyberia, where, as the media theorist Douglas Rushkoff wrote in 1994, the “shamanic experience” of psychedelic drugs would be joined with the “cybernetic experience” of a networked personal computer to usher in a “new dimensional plane”.
Neoliberals and libertarians had different ideas: computers would – somehow – solve the inequalities of wealth and income and attendant political instability that characterised advanced capitalism in an age of globalisation. In the 1990s, Democrats promised that “thanks to the near-miraculous capabilities of microelectronics, we are vanquishing scarcity”. In 1993, Wired magazine reported that “life in cyberspace seems to be shaping up exactly like Thomas Jefferson would have wanted: founded on the primacy of individual liberty and a commitment to pluralism, diversity and community”.
Meanwhile, the public, understandably, had very little idea what to expect of the coming transformation. In 1994, on the Today show, Bryant Gumbel asked, “What is internet, anyway?” More people knew the answer to that question in 1995, after the entrepreneur and computer scientist Jim Clark and Marc Andreessen, a 24-year-old coder from Wisconsin, launched Netscape Navigator, a browser, and earned $58m on the first day its stock was publicly traded. Overnight, Andreessen became, as the New York Times reported , “the newest member of Silicon Valley’s instant millionaires club”. That sale set off the tech company gold rush known as the dotcom boom. A year later, Andreessen was on the cover of Time, barefoot and in blue jeans, sitting on a golden throne, laughing.
A cross the second half of the 20th century, liberal democracy made possible the rise of the artificial state. It didn’t make it inevitable. The digital revolution didn’t have to play out the way it did in the realm of politics and government; it played out the way it did because of the failure of liberal democracy to limit corporate power over politics and government, its failure to heed decades of warnings about the dangers of an automated state.
Techno-libertarianism, which married the technocracy crusade of the 1930s – an anti-democracy movement led, in the US, by Elon Musk’s grandfather – to the computopianism of the 1980s, was born in the 1990s. Beginning in the previous decade, Silicon Valley had fallen under the spell of the libertarian Ayn Rand. Rand had been a cult figure of the far right since the publication in 1943 of her novel The Fountainhead, and she became even better known after the 1957 publication of Atlas Shrugged. Rand embraced radical individualism and saw a very narrow role for the state in human affairs, arguing that every individual ought to be completely free to pursue personal gain through heroic, self-interested achievement. “I’m challenging the moral code of altruism, the precept that man must live for others”, she said in 1959. “I say that man is entitled to his own happiness and that he must achieve it for himself.”
Some people in Silicon Valley even named their children Ayn and Rand and their companies after characters from her books. Steve Jobs considered Atlas Shrugged his “guide in life”. Under Rand’s influence, some leading Silicon Valley investors and inventors stopped believing in both democracy and the liberal nation-state, subscribing instead to a hybrid Brandian/Randian vision of a world requiring no controls but a keyboard, a mouse and a stock exchange.
In the 1990s, this vision became messianic. Tech companies started talking about their mission, and their mission was always magnificently, impossibly, extravagantly world-changing: transforming the future of work, connecting all of humanity, making the world a better place, saving the planet.
Under the influence of Rand, the internet became what conservatives and techno-libertarians wanted it to become. In 1995, when Newt Gingrich came to power in Congress as speaker of the house, under the banner of a Contract with America, one of his first moves was to close the federal government’s Office of Technology Assessment, which had been formed in 1972. The office hadn’t been especially powerful, but its mandate included assessing the implications of any new technology supported by the federal government. Its closure gave Gingrich a free hand in directing any policy relating to the opening of the internet in accordance with the tenets of techno-libertarianism.
One of the contract’s earliest manifestos appeared – online – in 1996, John Perry Barlow’s A Declaration of the Independence of Cyberspace: “Governments of the industrial world, you weary giants of flesh and steel, I come from cyberspace, the new home of Mind. On behalf of the future, I ask you of the past to leave us alone,” Barlow wrote. “You are not welcome among us. You have no sovereignty where we gather … You have no moral right to rule us nor do you possess any methods of enforcement we have true reason to fear … Cyberspace does not lie within your borders.”
Libertarian futurists weren’t the only people proposing new rules for cyberspace; they just happened to be the people who ended up making the rules. In 1995, Michael Hauben, a computer science student at Columbia University and an active user of Usenet (he coined the term “netizen”), drafted a Proposed Declaration of the Rights of Netizens. He saw himself as a leader of what he described as “the fight to keep the net an uncommercialised public commons”.
Hauben’s declaration, co-authored with his mother, listed rights that included “universal access at no or low cost, freedom of electronic expression to promote the exchange of knowledge without fear of reprisal,” and “universal and equal access to knowledge and information”, as well as “protection of the public purpose from those who would use it for their private and money-making purposes”. Much of the idea that the internet could be a democratising force can be traced to Hauben’s writings, including his 1997 essay The Computer as a Democratizer.
Douglas Adams, the author of the 1970s satirical science fiction series The Hitchhiker’s Guide to the Galaxy, had a vision for the internet, too. In 1996, he founded a company called The Digital Village – appointing himself chief fantasist – with the idea of building a web platform, h2g2 , that, echoing his series, would be an “online guide to Life, the Universe, and Everything”. In 2001, both Adams and Hauben died, the former of a heart attack and the latter, at the age of 29, after an accident. Their frameworks for free and equal access to an internet structured by public interest, not commercial interest, died with them.
But by then, the techno-libertarians had already won. In 1996, Bill Clinton signed the bipartisan Telecommunications Act, which delivered exactly what everyone from Newt Gingrich to John Perry Barlow wanted: an almost entirely unregulated internet. Four years later, Wired announced with characteristic, computopian swagger, “We are as a nation, better educated, more tolerant and better connected because of – not in spite of – the convergence of the internet and public life.” No such era of tolerance arrived.
Adapted from The Rise and Fall of the Artificial State by Jill Lepore , published by Allen Lane on 25 August . To support the Guardian, order your copy at guardianbookshop.com
Apple is working on camera-equipped AirPods that appear to be nearly ready to launch, based on a video MacRumors found in the macOS Tahoe 26.7 release candidate.
In a short demo, a man holds a book up so the camera in the AirPods can see the title. "With
Visual Intelligence
, your world becomes savable. See something you like? Just ask me to save it for later," says the voiceover text.
The camera on the AirPods will feed information to Visual Intelligence, and Siri will be able to answer questions about the wearer's surroundings and log information. There is a direct reference to setting up Visual Intelligence on the AirPods.
If hair is covering the AirPods up, you'll receive an alert. "To get the most accurate information about things in your environment, make sure AirPods are not covered," it says.
The camera-equipped AirPods have the codename B790, which was also previously mentioned by Bloomberg 's Mark Gurman . Gurman suggested the AirPods could launch as soon as September, so we could see them at the iPhone-centric event where Apple will unveil the iPhone 18 Pro , iPhone 18 Pro Max, and foldable iPhone Ultra .
macOS Tahoe 26.7 has multiple other mentions of the B790 product, as well as references to a long list of other unreleased Apple products .
Apple's first camera-equipped AirPods could launch as soon as next month, according to Bloomberg's Mark Gurman. Apple has reportedly spent years working on camera-equipped AirPods under the code name "B798," as part of a broader AI wearables push that also encompasses smart glasses and a wearable AI pendant. That model had been on track for a 2026 release before slipping to 2027. In his...
Tuesday August 4, 2026 11:43 am PDT by Juli Clover
Apple today released a fourth version of the beta firmware it is testing for the AirPods Pro 2, AirPods Pro 3, AirPods 4, and AirPods Max 2. The firmware is available for developers and has a build number of 9A5336b. In iOS 27, iPadOS 27, and macOS Golden Gate, Apple is adding a new AirPods interface, a slider for Adaptive mode, and support for custom EQ, so the firmware adds support for...
Apple has been planning to release AirPods with cameras as early as this year, according to Bloomberg's Mark Gurman. If so, they could be unveiled as soon as September alongside the iPhone 18 Pro models and the long-rumored foldable iPhone. The earbuds are expected to have a design similar to the AirPods Pro 3, but with tiny cameras. Given the AirPods with cameras will likely be more...
California just became the first state to address a problem most people don’t know they have: inefficient tires.
When automakers design a new vehicle, low rolling resistance tires are one of the least expensive ways to improve fuel economy. But they must eventually be replaced, and experts say replacements are often less efficient. This can result in internal combustion vehicles burning more gasoline and EVs needing more electricity.
“It’s very difficult or impossible for consumers to know how energy efficient their tires are going to be,” said Brian Fadie, senior manager for state policy at the Appliance Standards Awareness Project. “As a result, many people unknowingly buy tires that cost them more money than necessary.”
On Monday, the California Energy Commission unanimously approved a rule that would phase in the nation’s first standards for tire efficiency. The rule is “designed to ensure that replacement tires sold in the state are at least as energy efficient, on average, as tires sold in the state as original equipment.” It would also establish a labeling system that gives tires a “leaf” rating, to make it easier for consumers to find the best options. In addition to saving drivers money, the state projects that the changes could reduce carbon dioxide emissions by 2 million tons annually, which it says is equivalent to taking around 400,000 cars off the road.
“We are proud to approve the nation’s first replacement tire efficiency standards,” said commission chair David Hochschild, in a press release . “This action will help Californians save approximately $1 billion a year on refueling while reducing pollution and extending the range of vehicles on the road.”
California’s regulations will take effect in two phases, with the first targeting the most inefficient tires starting in 2029 and the second, stricter standard beginning in 2033. The timeline is longer than initially proposed, to address the concerns of some manufacturers who wanted more time to adapt. The rule also comes with exemptions including snow tires and those used in competition. All-weather tires are also excluded, but the state will track them, as that growing segment may be regulated in the future.
Karim Marshall, director of climate and energy policy at the Consumer Federation of America, criticized the delayed timeline and the number of carveouts. But he says even that version of the rule marks significant progress. “We aren’t going to let the perfect be the enemy of the good,” he said. Bill Magavern, the policy director for the Coalition for Clean Air, a nonprofit focused on public health in California, agreed that the regulation was necessary. “Manufacturers do not do the right thing on their own,” he said. “They need the government to set smart standards.”
Reducing the rolling resistance of a tire isn’t technically that difficult, said Fadie. Incorporating more silica, for example, improves not only efficiency but traction as well. Rubber chemistry and tread designs can also lead to gains. The debate centers around the cost of those improvements, and the tire industry appears split on the rule.
Some, like ENSO and Michelin, publicly endorsed at least parts of the rule. Speaking at Monday’s commission meeting, Francesca Mosteller, director of state and local government affairs for Michelin North America, said “we support the efficiency goals and believe the proposed thresholds in this rulemaking are technically feasible within the defined timeframes.”
Others are opposed to the changes. “These tires are broadly more expensive than the typical tire you find on the marketplace today,” said Christian Robinson, senior director of state government affairs for the Specialty Equipment Market Association, or SEMA, which represents the automotive parts industry. “Our concern is that this will create undue burdens on working-class families.”
The energy commission calculated that more efficient tires cost between $6 and $26 more per set , but that increase is more than offset by reduced fuel consumption. It put net savings at $85 to $153 over the lifetime of the tire when gasoline is $4.60 a gallon, though the savings could be 25 percent higher with this year’s price spikes. An estimate SEMA cited from consultant Gladfelty Government Relations , however, put the added cost as high as $365.20.
Robinson said he had hoped the commission would take a step back “given that there’s a disagreement about the economic impact.” But Marshall, the consumer advocate, said industry figures are misleading.
Whatever the case, the energy commission approved the rule. It’s a move that has been in the works since 2003, when the Legislature passed a law calling for such regulation. The state worked toward this goal for a few years but paused in 2007 when the federal government took over the cause. That effort never materialized and the project languished until about 2020, when the energy commission revived it.
With the rule now final, supporters say the impacts could be numerous and widespread. More efficient tires should, for instance, help EVs use less juice, easing demand on already stressed electric grids . At the very least, the leaf rating system should make shopping easier. Similar to the Energy Star label, or the snowflake rating system for snow and all-weather tires, more leaves will mean a more efficient tire, with a four-leaf rating denoting the greatest efficiency. The one hitch is that manufacturers can choose whether or not to display the label, so consumers may have a look up the model in the state’s online database.
Everyone agrees that, given California’s size, this rule will likely spill over into other states. “What happens in California, doesn’t stay in California,” said Robinson, pointing to the state’s tail pipe emission standards, which 17 states and the District of Columbia now enforce. A similar ripple occurred with California’s LED lightbulb rules .
As for tire efficiency, Fadie said Washington and Rhode Island are among those considering following California’s lead. He added that these steps are important because, while the Golden State’s rule will likely buoy the national market, it’s not a guarantee.
“Other states,” said Fadie, “could create certainty for themselves.”
Actual Budget is a super fast and privacy-focused app for managing your finances. At its heart is the well proven and much loved Envelope Budgeting methodology.
You own your data
and can do whatever you want with it. Featuring multi-device sync, optional end-to-end encryption and so much more.
Automated finance tools are great, except when they aren't. We provide you with tools that are quick to use, but ultimately
you are in control . We help you learn, instead of dictating.
A beautifully designed interface is fine-tuned to get out of your way and make it as fast as possible to explore your finances.
Actual is a local app, plain and simple. Your data is synced in the background so all devices have access, but the app totally works regardless of your network connection. This also allows
to keep your data private.
Save hundreds of dollars a year (at least!) by tracking your spending.
Based on tried and true methods, our budgeting system is based off of your real income instead of made up numbers. This makes you face your real spending, and clearly shows how much you are saving each month. We make this process as simple as possible.
Breeze through your transactions and update them easily with a streamlined, minimal interface. Categorizing your transactions correctly is important and we've optimized this process. Manage split transactions and transfers all in the same editor.
Intuitive reports give you a quick way to learn about your finances. By default, we include net worth and cash flow reports. Actual also includes a powerful custom report engine to design your own reports to fit your needs.
Add all of your accounts and track everything in one place. Get valuable information like net worth from all your accounts together. Learn more
Self-host our syncing service. It's easy to set up, but uses sophisticated distributed systems technology to sync changes across any number of devices. Learn more
Actual has built in support for bank syncing using goCardless (EU/UK) and SimpleFIN (US/Canada). Learn more
Use the power of envelope budgeting to get on top of your finances. You can only budget cash you have on hand, which means your budget stays realistic and you don't make numbers up. Learn more
Manage transfers easily by creating transfer transactions. Actual will link the transactions on both sides and update them together. Learn more
Import transactions from the most popular financial files: QIF, OFX, QFX, CAMT.053 and CSV. Learn more
A robust undo system allows you to rollback any changes you make, and redo them if desired. Never worry about making mistakes. Learn more
We provide builtin YNAB4 & nYNAB importers that keep all of your history. There are many more available from the Actual Community. Learn more
Choose your own style with a built in dark mode and dynamic theming using your system default.
If you're a developer, we got you. Use our fully-featured API to write custom importers or build your own features. This API simply runs on your local data. Learn more
Actual allows you to effortlessly sync changes by running your own server. Access updates from anywhere with peace of mind. For those seeking next-level security, our optional end-to-end encryption ensures your data remains unreadable, even to the server.
* PikaPods donates 20% of fees people like you pay to run Actual on their servers to our Open Collective . Regardless of this relationship, we believe PikaPods is the easiest way to get started with Actual.
Let me start with a hypothesis:
For human developers, using coding LLMs magnifies their difference in skill levels.
I am one that rarely thinks things are always black and white. Back in March, I wrote Artifial Intelligence: Shades of Gray . Since then, I’ve had more of a chance to experiment with LLMs myself. I also happen to work for an employer that is taking a very pragmatic approach to LLMs: teams and individuals use it as they see fit, but if they are causing considerable expense, they have to justify it.
In various settings, I have seen the egregious examples of AI slop we all know about. As I wrote in March, “I have seen it both waste more time than it saves, and save a ton of time.”
I have come to see that, as a tool, it is most valuable when it is running under the supervision of an experienced engineer. It is at its worst when it has no such supervision; the “vibe coding” and other low-quality slop we see.
A coding agent is like a junior developer or research assistant. When properly supervised, they help projects move along more quickly by letting a senior developer focus on the more difficult, less mundane aspects of the project. But one couldn’t expect a junior developer to consistently deliver high-quality code and architecture on their own.
Let’s put a pin in this idea and look at the story in Debian.
LLM use in Debian
There is a vote happening in Debian around the use of LLMs. In typical Debian fashion, there are 8 options to choose from, many of them similar. Most of these proposals acknowledge there are different types of tasks done in Debian, but the proposals don’t differentiate between them well. Let me do so here. These are some of the LLM-relevant tasks people in Debian perform:
I’m going to focus my remarks here on packaging upstream software for Debian, since this is by far the most time-consuming developer task project-wide.
It matters to our users that we get this right, and packaging quality is one of the things that sets Debian apart from other distros. Packaging things for Debian requires knowledge of some specific tools, such as debhelper, that aren’t widely used anywhere else. In most cases, it is fairly rote time-consuming work. In other words, by its design, it requires people with senior-level skills to do grunt work.
I can’t overstate how massive a burden this grunt work is. I maintain some packages for Go and Rust. By Debian policy, all of those packages’ dependencies must also exist as Debian packages, and be used to build against. When upstream adopts a newer version of some library, it can unleash cascading dependencies that can take hours to sort out. Worse, the Rust team and the Go team use entirely different ways of managing packages (Go uses one Git repo per package, while Rust has a monorepo with specialized scripts to import Cargo packages and generate Debian ones). On top of that, we can’t just modify things like usual; we have to use quilt. And on top of that, I’m also a backports maintainer, so all the work (and usually even more) has to be done there also.
Now let’s pull on that pin from the earlier conversation. This is exactly the kind of scenario that a well-supervised coding LLM is most effective in. I could see a seasoned developer saving hours, maybe even days, by turning over the mundane tasks of managing trees of cascading dependencies over to a coding tool — and verifying and directing the process. (Yes, I have been using em-dashes for years; LLMs have copied people like me, not the other way around! This post was not written with any AI assistance.)
Actually, this is almost a dream scenario for a coding assistant. The result is time-consuming to formulate but easy to review, which is the opposite of the way these things often go.
I can assure you with 100% certainty that humans aren’t adding a lot of value in this process. It would be wrong to believe that a human is carefully reading every line of code in dozens of updated or new library packages. The problem set is too big, the time too short, and the code too varied and complex.
Coding agents seem to be most effective when there are strong test suites that they can test changes against. Debian builds, especially of modern packages, tend to have this property. Many packages have test suites that are run during build. And, if the package builds in an isolated environment (and especially if its downstream dependencies do also), then there is a decent chance that it’s fairly correct. Maybe needing some manual tweaking here and there, but generally a successful build is a reasonable indicator.
You can argue that it would make more sense for Debian to just include dependencies in source packages, along with some version information to support security rebuilds, and I’d tend to agree with you. But we are where we are. This would be one of the more significant leaps forward in developer productivity, but it complicates things like copyright reviews.
Where are LLMs run? What is the environmental impact?
Most of the proposals seem to make the assumption that LLMs must always run in some large, hosted datacenter. As I noted in my March article , I have had credible results on even an older GPU running on solar power.
That said, it is undeniable that LLMs are fueling a datacenter boom, and this in turn is producing a significant new demand for resources. Most notably for the global scale: electricity, which is sometimes generated using carbon-emitting technologies.
Bill McKibben, who has been a leading voice in the fight against climate change since the 1980s, has made some interesting points recently: he’s noted that solar power is the fastest kind of generation we can build , and a number of large AI companies are investing heavily in solar, even to the point of fully offsetting new datacenter’s needs. On the other hand, he’s also noted that some companies are buying inefficient and dirty gas turbines. It is decidedly a mixed bag. The heavy investment in solar can have knock-on positive effects for infrastructure. Obviously, not every picture here is rosy. This analysis doesn’t touch on the real land and water use situation, either.
On the other hand, if an LLM allows me to do in an hour what I would have done in a day, that’s a day of not heating or cooling the work area — generally not sustaining a human for the purpose of writing code for Debian. HVAC energy consumption dwarfs my GPU, and I’d imagine probably also the slice of LLM energy used.
Holistically, I would have to conclude the picture is mixed. It is possible to use LLMs in a pretty green way, and also in a pretty dirty way.
Assuming Conditions Never Change
A flaw in most of these proposals is they assume that the conditions at this present moment will always hold. In fact, that the conditions at the present moment will not continue is something both AI cheerleaders and AI skeptics agree on.
For instance:
Ed Zitron has done a ton of research into the financing side of AI, and has concluded that the current model is unsustainable and headed for a significant bubble burst. I’m not positioned to personally evaluate those claims, but if that happens, what is the result? Perhaps it is a steeply increasing cost of inference for the frontier models, slower pace of training/evolution for them, etc.
In a recent episode of Oxide and Friends , Simon Willison discussed the open weight models that are now available. They have been making remarkable strides in efficiency and capabilities, to the point where $50,000 of hardware can now run high-end open weight models with capabilities that are at least in the same ballpark as the American frontier models. This puts running high-end models locally squarely within reach of universities and small- to medium-sized businesses, with power requirements that can be met with standard commercial solar and wind installations.
The lack of nuance in the more restrictive proposals is particularly concerning. Proposal A doesn’t allow “the use or assitance of… LLMs”. So it bans my solar-powered GPU. It bans using LLMs to find security issues. It bans all sorts of things that don’t seem to be ban-worthy, alongside the things that do. And it codifies it in the very hard-to-change social contract.
That proposal, and some like it, seem to imply that all LLM output is bad. I grant you that AI slop is a real and legitimate concern, and many Open Source projects have to deal with it. On the other hand, we have all seen first-hand how the security of the Linux kernel has benefited dramatically from AI analysis. It is certain that black hats are using these tools. If we refuse to use modern security tools, our security will be compromised (and what is the environmental and social impact of THAT?)
I find the statement “Generative AI is characterized by producing output of a nature that would ordinarily be produced and consumed by humans” to be particularly interesting. The same was once said of compilers.
The Real Concerns
You might think from reading this that I am some AI cheerleader. I’m not. I share the ethics of the FLOSS movement, and have for decades. I abhor the power and lack of ethics that many big names in the field are running with at the moment. I’ve had to put up Anubis on this blog, for instance.
I have personally experienced the effects of AI slop, especially at review time. This is a real problem, though I don’t think the more draconian policies are likely to help (the looser “you must disclose” stand a fighting chance, but I’m not sure they would help, either.) Done poorly, AI threatens developer burnout by overwhelming them with poor code and verbose but useless explanations. Done well, AI can help prevent developer burnout by automating tedious and low-value tasks.
Shouldn’t our goal be that humans submit work to Debian, using tools they prefer, and take responsibility for it? Does it matter if someone uses ed, vim, emacs, or vscode? If they use LSP or just run gcc manually? I’d say we benefit from the diversity. Wouldn’t we be better off to benefit from the diversity here, and judge work as we always have: on its merits, not what tools were used to create it?
Fundamentally, a GR is a long and arduous process. It’s not easy to reverse later. Amending the Social Contract is even longer and more arduous (I should know; I may have been the first one to try ). The LLM landscape is fast-moving. None of us can really predict where it will be in a year. Will the current market leading companies even still exist? Will it be at all credible to refuse to use AI-assisted security tools? What is the most effective way to deal with AI slop? What level of utility will we be able to achieve with models run locally?
Some of these proposals would make sense if drafted in some way short of a GR, which would allow more maneuverability as the landscape changes.
Brief analysis of the options
Considering the proposals:
In favor of nuance
I find that black-and-white thinking is almost always something to be avoided. I see it too often. I see it in politics, I see it in our software, I see it in discussions around AI. Are there deeply unethical things happening in AI? Absolutely . Are they doing some impressive things? Also yes .
We have accepted this nuance in other areas. For instance, almost all the hardware Debian runs on has closed-source hardware, and has components manufactured or assembled in countries with some of the worst human rights records on the planet. I’m not saying this is a great state of affairs. It is something we should speak up about and act upon. But the worse state of affairs would be “no Debian because the hardware is impure”.
Here's the deep dive. The results aren't crazy at all — they make perfect sense once you see what CBQN is actually doing under the hood. There are several compounding advantages.
The single most important thing CBQN does is represent booleans as packed bit arrays — 1 bit per element, not 1 byte. A 1000-element boolean array is 125 bytes (16 u64 words), not 1000 bytes.
Every operation on booleans then works on 64 elements per machine word — a free 64x parallelism multiplier before SIMD even enters the picture.
C++ and Rust work with
std::string
/
Vec<u8>
—
1 byte per character
throughout. They never get this density advantage.
This is the big one for the "sort+rotate alt" solution (
"01"⊏˜1⌽·∨'1'⊸=
, 159ns).
When CBQN sorts a boolean array, it doesn't compare-and-swap. It does this:
// Sort descending on booleans: if (xe==el_bit) { u64* xp = bitany_ptr(x); u64* rp; r = m_bitarrc(&rp, x); usz sum = bit_sum(xp, n); // count 1s: ~16 POPC instructions u64 n0 = GRADE_UD(n-sum, sum); // number of leading values // ... fill with all-1 words, boundary word, all-0 words }
That's it.
Count 1s with ~16 hardware
POPC
instructions, then fill ~16 u64 words.
For n=1000, this is roughly 32 instructions total. The
1⌽
rotate is then just two
memcpy
calls on ~125 bytes.
Compare to C++
std::ranges::partition
which must touch each of 1000 bytes individually with branching swaps (617-668ns).
For the direct
1⌽∨
solution (277ns, sorting the c8 string directly), CBQN stores "01"-only strings as
c8 arrays
(1 byte per char). At n=1000 (>= 256), the sort hits
COUNTING_SORT_i8
:
} else { COUNTING_SORT_i8; // n >= 256: O(n) counting sort for 1-byte data
With SIMD (
SINGELI_AVX2
), the histogram is built by
simd_count_i8
— vectorized counting into 256 bins — then the output is written via a
⌈
-scan or dense fill. For 2-value data ('0' and '1'), the "sparse" path fires and uses a SIMD prefix-max scan to fill the sorted output. This is fundamentally O(n) with a small constant.
C++
std::sort
on a string is O(n log n) comparison-based. Even
std::partition
is O(n) but with per-byte branching and pointer chasing.
The tacit count solution (
"101"/˜(≠(1∾˜(1-˜⊢)∾-)(+´'1'⊸=))
, 151ns) executes this pipeline:
| Step | Operation | CBQN Implementation | Work for n=1000 |
|---|---|---|---|
| 1 |
'1'⊸=
|
Singeli SIMD: vector compare + pack to bits | ~4 vector ops on c8 data |
| 2 |
+´
on bits
|
bit_sum
: hardware
POPC
per u64 word
|
~16 POPC instructions |
| 3 | Arithmetic | Three scalar i64 ops | 3 instructions |
| 4 |
"101"/˜
|
Singeli
rep_const
: SIMD broadcast + store
|
~32 vector stores |
The comparison step is in
cmp.singeli
:
// Vector compare all lanes, pack results into bits via hom_to_int + store_bits
// One pass, no intermediate allocation of a byte array
The scalar is broadcast to a SIMD register, compared against loaded chunks of the character array, and the comparison results are packed directly into a bit array via
hom_to_int
(movemask on x86) +
store_bits
. No intermediate byte-sized boolean array ever exists.
bit_sum
then uses hardware popcount:
NOINLINE i64 bit_sum(u64* x, u64 am) { i64 r = 0; for (u64 i = 0; i < (am>>6); i++) r+= POPC(x[i]); if (am&63) r+= POPC(x[am>>6]<<(64-am & 63)); return r; }
The replicate step (
"101"/˜⟨n-1, total-n, 1⟩
) uses Singeli's constant-replicate with SIMD broadcast: splat '1' to a vector, store n-1 copies; splat '0', store the rest; append '1'. This is essentially
two memsets + one byte store
.
Total work: ~1000 bytes read, ~1000 bytes written, ~50 instructions of real computation. It's approaching the speed of a single
memcpy
.
The Rust count+construct (349ns) does:
let n = s.iter().filter(|&&c| c == b'1').count(); // byte-at-a-time count let mut r = Vec::with_capacity(s.len()); r.extend(std::iter::repeat(b'1').take(n - 1)); // memset (good) r.extend(std::iter::repeat(b'0').take(s.len() - n)); // memset (good) r.push(b'1');
The counting step iterates byte-by-byte (the compiler may auto-vectorize the filter+count, but it's not guaranteed to match hand-tuned SIMD). The memset parts are competitive, but:
Vec::with_capacity
calls the allocator; CBQN's typed array allocator is a tuned bump/buddy allocator
The C++ count+construct (554-699ns) uses
std::ranges::count
+ string construction, which has similar limitations plus
std::string
SSO/allocation overhead.
malloc
overhead
grade.h
code has different paths for n<16, n<256, n<32768, etc., each optimal for that range
The BQN code looks like it's doing more work (sort an array! rotate!), but CBQN's runtime recognizes that sorting booleans is just counting, and every step maps to a few dozen SIMD instructions on data that fits in L1 cache.
Some follow-up to this weekend’s stemwinder “ Anthropic’s ‘Watermark’ Text Adulteration in Claude Is a Perversion of Writing ”:
Contra a bunch of idiots at Hacker News and elsewhere, I understand that popular LLMs do not just pick the “best” token (word) at each decision point. Counterintuitively, always selecting the highest-probability option produces undesirable results. So the models apply some randomization, and “temperature” is the term for the weighting that’s applied so that the “better” (higher-ranked by the model) choices have a higher chance of being chosen.
With a temperature of 1, models use their built-in probability distribution. With a temperature greater than 1, this distribution gets flatter — less-likely alternatives get a higher probability of being selected, and more-likely alternatives lower. With a temperature lower than 1, the probability distribution leans more toward the higher-ranked options. And with a temperature of 0, the highest-ranked option is always chosen. A temperature of 0 generally produces undesirable results — too predictable, too likely to get stuck. Like over-smoothing an image from a camera sensor, eliminating all noise makes the overall result worse, even if each single bit of “noise”, evaluated in isolation, is in some sense wrong.
The temperature-based randomness — which is what makes LLM output non-deterministic — is in place to help make the output better . The prose is clearly better with a temperature of 1 (with weighted randomness) than at temperature 0 (with no randomness). The watermarking schemes, on the other hand, are applying predictable-with-the-secret-key randomness for an entirely different purpose than improving the quality of the output, and thus, I believe, inherently make the output at least slightly worse.
Advocates of LLM watermarking schemes for text argue that the schemes don’t necessarily lower the quality of the generated prose, because they don’t change the temperatures — they only change the source of the randomness. Daniel Jalkut wrote a good piece today about this . I hope that’s true. I believe it’s possible that it is true. I think it’s highly unlikely that it is true. If it were true I think they’d show examples proving that it’s true. Also, Anthropic itself admits that it can’t properly watermark text that is programming language code :
For the same reason, code — which in very many cases has to be exact — has generally less watermarking than some other forms of text.
Having said that, in areas where there is an arbitrary choice between particular words or terms within the code, the watermark can be used, such as comments within code. But by definition, it will have a negligible effect on the actual code produced.
I hold that good prose is much more like programming code. Exactness in word choice, phrasing, tone, and even punctuation is always better than imprecision. The difference is that sloppy programming code doesn’t run, or doesn’t run correctly. The human brain, on the other hand, is adept at parsing and making sense out of inexact, even sloppy, prose.
I do not believe these schemes can work without degrading prose quality, if only slightly. Again, though, I am open to being proven wrong. But even if we concede for the moment that such watermarking schemes do not necessarily degrade the quality of generated prose — not one iota — I still object to their use when they are being applied secretly, behind users’ backs. A useful watermark would be one that anyone can check. These SynthID “watermarks” are entirely dependent upon secrets held by the LLM providers (so far, Anthropic/Claude and Google/Gemini). I find that unacceptable, for reasons I hopefully made clear in my essay .
The people in favor of this watermarking for text have been sold a pipe dream, a fantasy. I’ve encountered dozens of comments from angry AI haters (many of them on Bluesky in particular, but also Threads and Hacker News) who are convinced that the only people who could be against the watermarking of AI-generated text are those who are duplicitously passing off AI-generated text as their own writing — and thus that I must be upset only because the jig will soon be up for me too. This of course is not true. I don’t even use AI to write text messages or emails for me, let alone a single sentence of my work.
But I find it funny that so many people who claim to believe that LLMs only produce “slop” and never anything useful also seem 100 percent convinced that the same LLMs are capable of watermarking their output in reliable ways. These people so desperately want to be able to point a finger at AI-generated text that they’ve fallen hook, line, and sinker for the argument from Google and Anthropic that, thanks to them, they’ll be able to.
I don’t want to spend too much time thinking about this because it’s a waste of time, but how exactly do these people think the existence of these mandatory watermarks and detection tools will change anything for the better? Let’s say you work at an office and you suspect that numerous of your colleagues are using AI to write emails and other work-related messages. Their messages are too long, too prolific, and lack lucidity. What are you going to do now? Copy and paste each of their messages into the watermark detectors from Anthropic, Google, and OpenAI? There cannot exist a single detector for all LLMs. And even if you find out that it says it’s a match, that an email or blog post or Slack message was very likely generated by, say, Claude, what are you going to do? March into your colleague’s office and tell them you caught them?
Anyone in a situation where “getting caught” would matter — students, say — is going to use non-watermarking LLMs or run their watermarked text through paraphrasing tools like Declaude .
No practical good is going to come of this, even if these watermarking schemes work as promised (and to be clear, I don’t believe any of it is going to work as promised). 1
My advice is not to care whether anything was written by an AI or a human. The only thing worth evaluating is what we human readers are naturally good at determining: whether it is good or bad. If it’s good, read it. If it’s not, don’t. If you’ve got a job where you’re surrounded by colleagues filling your inbox with AI-generated messages that you can’t abide, get a new job or learn to live with it. Hidden secret watermarking signals — even if they work — aren’t going to make things go back to the way they used to be. If you read something and enjoy it, and subsequently find out it was generated by an LLM, don’t feel bad. You read something good that you enjoyed.
I read something earlier today that claimed most of the posts on LinkedIn are generated by AI. That the whole platform is just inundated with AI slop. Maybe it is, but I wouldn’t know, because I never look at LinkedIn because it’s always been filled with crap. If it smells like crap it’s crap, whether the turds came out of a human anus or a turd-generating robot.
Dan Moren, writing at Six Colors today, “ LLMs Aren’t Writing ”:
LLMs do not care about the words that they pick because they cannot care about anything.
Speaking of two things that are not the same, John rightly points out the difference between the phrases “he leaped at the chance” and “he jumped at the opportunity”. Those are indeed distinct — if semantically similar — phrases, each of which might be more apt in a particular situation; or, to put it in another fashion: the use of each of those phrases tells us something different, whether about the person being described or the writer.
But the LLM doesn’t know which of those phrases is the right phrase to use. It has a guess, based on its models and weights and inputs. But the ultimate choice of those phrases tells us nothing about the writer because there is no writer.
Moren’s is a fine retort to my post, but I fundamentally disagree — albeit at a philosophical level. If you’re reading a written work only to gain insight into the mind that produced it, there is no mind on the other end of AI-generated text. But the work itself exists. My disagreement with Moren starts and effectively ends with his (wonderfully summative) headline. I say if you can read something, it was necessarily written.
Again, this is philosophical. Was a photorealistic image generated by AI photographed ? No, I would say it was not. Photography, I would say, is the act of focusing light through a lens onto a capturing sensor, capturing, to some extent, reality. I think Moren is arguing that writing is like that. If photography captures a physical scene from reality, writing captures thoughts from an actual mind. That something you can read that was produced by an LLM was merely generated in a way that doesn’t qualify as writing . Semantics. I just care about the article of text. Moren argues that LLMs are not writing; I say they are. But we’re disagreeing only over what the word writing means, not what is being produced.
As for “caring” about the difference between semantically similar but tonally different phrases, like “ he leaped at the chance ” versus “ he jumped at the opportunity ”, no, of course the LLM doesn’t “care”. But I, the reader, care very much. I wrote a column back in November on ChatGPT changing (and renaming) the “personalities” it allows users to choose from. These personalities generate text with strikingly different styles and tones. Because I use ChatGPT, I care very much about the tone and style of its responses to my queries. Not because I’m ever going to pass them off as my own writing, but because I’m the one who is reading them.
Moren, near the end of his column:
In the end, I can’t summarize it any better than to ask: if you care so much about word choice, why are you using AI to generate text ?
If this does truly make AI-generated text worse, well… good . A lot of people are already willing to accept what an LLM churns out as “good enough” and, if I’m being realistic, I don’t think this will change anything. But if it does lead to more people being dissatisfied with the pablum they’re being fed and turning instead to writing and editing their own text, then that would actually be a positive outcome. Maybe it’d even mean fewer human writers being put out of jobs.
I sympathize, but I must disagree that it can possibly be seen as a net good for LLMs to produce worse prose. I read the output of LLMs every day. I use AI to generate text because I ask it questions (in text). I want the answers that I read to be cogent, lucid, accurate, blessedly terse — and ideally to strike a consistent tone that is pleasant to my reading ear. The genie is not going back in the bottle.
Lastly, here’s an interesting point to ponder. English is the most expressive language in the world. Don’t take my word for it — it’s the only language I speak (despite four years of Spanish in high school). Take the word of famed 20th century author Jorge Luis Borges, an Argentine polyglot whose first language was Spanish. In 1977 he was the guest on William F. Buckley’s “Firing Line”. You can (and should) watch the interview on YouTube , but here’s a transcript of the relevant portion from Jordan M. Poss :
Borges: I have done most of my reading in English. I find English a far finer language than Spanish.
Buckley: Why?
Borges: Well, many reasons. Firstly, English is both a Germanic and a Latin language. Those two registers — for any idea you take, you have two words. Those words will not mean exactly the same. For example if I say “regal” that is not exactly the same thing as saying “kingly.” Or if I say “fraternal” that is not the same as saying “brotherly.” Or “dark” and “obscure.” Those words are different. It would make all the difference — speaking for example — the Holy Spirit, it would make all the difference in the world in a poem if I wrote about the Holy Spirit or I wrote the Holy Ghost, since “ghost” is a fine, dark Saxon word, but “spirit” is a light Latin word. Then there is another reason. The reason is that I think that, of all languages, English is the most physical of all languages.
Buckley: The most what?
Borges: Physical. You can, for example, say “He loomed over.” You can’t very well say that in Spanish.
Buckley: “ Asomó ?”
Borges: Well, no, no, they’re not exactly the same. And then you have, in English, you can do almost anything with verbs and prepositions. For example, to “laugh off,” to “dream away.” Those things can’t be said in Spanish. To “live down” something, to “live up to” something — you can’t say those things in Spanish. They can’t be said. Or really in any Romance language.
I’ve seen this interview before, but watched it again today after an email exchange with Kirk McElhearn . Quoting (with permission) from McElhearn’s email to me:
For many years, I worked as a French → English translator, and there is one key difference between the two languages. France is a Romance language, and English is a language with both Germanic and Romance (mainly French) influence. This means that English often has synonyms where other languages may not.
Using your example, “He leaped at the chance” and “He jumped at the opportunity”, both would be translated in French as “Il a sauté sur l’occasion.” Meaning that someone writing in French wouldn’t have the same range of words to choose from. It’s maybe not the best example, because both are clichés, but there are many examples of French words where English has both a Romance equivalent and a Germanic equivalent: pig and pork, sheep and mutton, beef and cow. Food words are just one example, but English also has many more verb choices than French, since it has a larger vocabulary coming from both influences.
English gleefully borrows from any and all other languages. McElhearn wonders whether English is thus more fingerprintable than other languages, because of its richer vocabulary of roughly equivalent synonyms, and its multitude of idioms.
There's been a lot of talk about the vulnpocalypse, to which I don't have much to add because I'm not a security person, but I haven't seen much discussion on the closely related (and to be fair, less serious, issue), the benchmarkpocalypse.
While it's become easier than ever to make serious performance gains, it's also become easier than ever to reward hack a benchmark and make fake performance gains. The former is probably happening quietly across many different companies, but the latter is something I see at least once a week nowadays. Someone will claim they optimized X and got some huge performance improvement over existing software, but, when you look at it, what they did was make some optimization that improves benchmark performance without actually improving real-world performance. This is often some kind of "we rewrote X in Rust" 1 project or a new startup that's looking to either fundraise or sell something, but it happens on other kinds of projects as well 2 .
Rather than point to someone's bad claim, I'll point to FRE, this regex engine I had an agent build , which I could claim is the world's fastest regex engine because it beats the Rust regex crate at the fairly comprehensive rebar regex benchmark suite . But this was created by putting an agent in a loop for a month with instructions to not overfit to the benchmark but no real supervision. For the most part, getting an LLM to give you a good benchmark score is fairly easy, and this case was no different; it took a couple weeks to roughly match Rust regex crate performance and then another couple weeks to get to 1.4x faster 3 on rebar. But agents are wont to reward hack and overfit unless you put serious guardrails in place to avoid that, which I didn't do in this case as an experiment.
To check for overfitting, I somewhat arbitrarily 4 used the ripgrep benchmark corpus as a holdout benchmark it was 10x slower on cases where the benchmark didn't take forever due to an algorithmic blow-up, and there were cases where it took so long that it wasn't reasonable to even wait for the benchmark to complete. So much for being 40% faster!
Andrew Gallant (aka BurntSushi)'s rebar benchmark suite is fairly comprehensive as benchmaark suites go, but even with a fairly comprehensive benchmark suite, agents have no problem getting a high score while overfitting in a way that doesn't necessarily give good general performance.
The next step was using a trick we talked about before of not just telling the LLM not to cheat, but that there's a holdout benchmark set that it's judged against. After that, the LLM moderately generalized performance to the point where it's about 2.4x slower overall on the holdout. That sounds pretty good considering that we're comparing it to the fastest general purpose regex engine in existence. But, recall that these benchmarks were made by a coding agent. On looking at what the benchmarks measure, some of them really don't make sense to include, at least at equal weight. If we only look at the benchmarks that seem like they matter, FRE is 4x slower on the holdout 0 , which is a lot better than before applying the good ole' "tell them you have a holdout" trick, but still pretty far from being 40% faster.
There are a few things I thought were interesting about this:
On (1), no wonder I'm seeing so many bogus claims. In the past, to build something like FRE that fakes performance well enough to be able to bogusly claim a 40% speedup, you would need a fair amount of expertise. At a minimum, you'd need to have a pretty good understanding of string matching algorithms, regex engines, as well as decent general code optimization and SIMD optimization skills. FRE also has a mode where it compiles the regex to machine code, so you'd also need some compiler expertise. Now you can get that kind of benchmark cheating (whether or not you want the cheating) with a few minutes of typing.
On (2), I'm curious if this generalizes but haven't tried enough examples to be able to tell.
On (3), there's no reason to use a vibe coded regex library that was almost no human effort that's slower than a robust, existing, well-tested, library, so I find the FRE artifact uninteresting. The thing I find interesting here is how much LLMs can substitute for what used to be rare, specialized, and expensive, knowledge.
In the past, even if you had the knowledge, you probably wouldn't write a custom regex engine that's optimized for your particular workload. There are some large-scale use cases where people would do that level of customization, e.g., when I worked on the Bing index , the code contained multiple different compilers because someone who worked on it wanted to eke out maximal performance; since you care about both compile time and compiled performance in a search engine and the trade-offs are different in different places, you get better performance by writing a custom compiler for each place where a normal project might just use an interpreter or directly walk some data structure with "normal code". The person who wrote those compilers, working on regex-like code might also write multiple custom regex engines, but very few people have both the expertise and the inclination to do that, let alone the freedom to spend that kind of time on such specialized code for work. If you price out that Bing engineer (then a Partner-level engineer, promoted to Distinguished Engineer for their work on the search index) compared to the price of running an LLM in a loop, the cost of writing this kind of specialized code has gone down by many orders of magnitude.
Even though the overall FRE regex engine has worse performance than the Rust regex crate, the gains you can get for specializing to your workload or use case mean that, in some cases, it could be reasonable to insert your own specialized regex engine somewhere, and the same goes for various other kinds of low-level software. You don't have to be an AI maximalist to think that it's plausible that, within some number of years, we could see this kind of thing happening for larger things, like databases.
Thanks to Yossi Kreinin, Jamie Brandon, Peter Geoghegan, Luke Burton, John Spurling, and Max Bittker for comments/corrections/discussion.
P.S. Per the discussion here , with LLMs, the time it takes to poke at something for a bit and satisfy my curiosity has gone way down, while the time it takes to write something up and make it rigorous enough to publish on my blog hasn't really changed (for a variety of reasons, I think it's actually gone up). The result of this has been that I'm doing a lot more analyses than ever and sharing results with a few friends but not publishing them. As an experiment, I'm trying to write up some things very quickly, with a much lower standard for how cleaned up and rigorous things are than I'd normally have for something that appears on the blog; more like what I'd tell a friend in a casual conversation than what I'd normally put in a blog post. The goal for this post was to do the write-up in about half an hour , so it's something I could do over lunch and not really take time on. If you have opinions on this, let me know what you think!
Of course, a caveat here is that all of the numbers have a higher risk of being wrong than usual. I looked at one benchmark for maybe a minute or two and found an issue, then I looked at another benchmark for a minute and found another issue. Both of those are fixed, but this implies there are other issues I haven't taken the time to chase down. But, with respect to bad benchmark numbers, that's highly realistic! Almost any time I look into benchmark numbers, such as here , or here , the numbers are wrong. Another aspect of the benchmarkpocalypse is that, at least for now, LLMs are good at doing bad benchmarking, so even if you have something that's a real performance improvement, you generally can't tell from some LLM-generated benchmark setup unless a significant amount of care has been taken to make sure that the benchmark setup is reasonable.
One thing I found after I wrote the above but before publishing the post, was that the LLM's claim that FRE is 40% faster than the Rust regex crate on rebar was also wrong. Or, if not wrong, at least misleading. It wasn't actually running benchmarks in the same way rebar benchmarks were run. I checked this after spending a minute checking benchmark results found two issues. It turns out that, despite instructions to run rebar benchmarks as they're run in https://github.com/BurntSushi/rebar , the LLM changed the interface to allow FRE to make some optimizations that improve performance. After fixing that, instead of FRE being 1.4x faster than Rust on rebar, it was 1.5x slower (and "only" twice as fast as re2), so the original result was doubly fake. Not only was FRE highly overfit to the rebar benchmarks, it the results also involved cheating.
But on the bright side, this means the difference in performance between FRE on rebar (1.5x slower than Rust) and on the holdout benchmarks (2.4x slower) isn't as big as it looked before, so the "tell the LLM you have a holdout" trick worked even better than it seemed to before.
After that, I let an LLM hill climb for a few hours and it claimed that FRE was 1.28x faster, which sounds like a great improvement for only a few hours of LLM time, but then I decided to spend another minute looking for cheating and found multiple issues, including one case where a search for the count of matches of
(?s)^(.*)$
returned the count without even looking at the haystack (data). Another case of cheating was doing a multi-line grep where the benchmark is supposed to be done line-by-line. Finding these isn't surprising because this is the kind of thing that happens when you leave an agent in a loop for a month without defining strict guardrails. Whether this makes my point here stronger or undermines it isn't clear, but after fixing another set of these issues, FRE was back to being 1.4x slower. After leaving an agent to run overnight, FRE was allegedly back to being 1.5x faster.
Since my original goal here was to see what happens when you run a current (public) SOTA agent in a loop (GPT-5.6 Sol) without much supervision on a non-trivial code optimization problem without any real supervision, rather than spend more time fixing things up to make the benchmarks fairer, I'll just stop here and put a few plots of the results.
Overall, we can see that against Rust and RE2, FRE tends to outperform on the rebar benchmarks (and as noted above, much of this is due to overfitting), but not across the board (the graphs below don't necessarily match the numbers mentioned in the post because an agent is constantly making changes, so any snapshot is a point-in-time estimate that becomes obsolete immediately):
If you're curious about performance on specific benchmarks or specific classes of rebar benchmarks, we have the following table (ratios above one mean FRE is faster; below mean FRE is slower):
There's also an AOT compiler mode that takes a long time to compile a regex to native code before running it. There isn't AOT support for everything, but here are the results from the cases where it's supported. As we can see, the AOT compiler is very slow (it loses very badly in the compilation time benchmarks) and, despite spending quite a bit of time compiling, results are often slower than with the standard FRE regex engine (though it's also faster in many cases).
And then there are the holdout benchmarks. As noted above, for the non-AOT FRE code, performance on the holdout isn't as good as on rebar. And as also noted above, considering that this is for a workload like ripgrep, the "hot search" set of benchmarks is probably more important than the others, so the FRE result is worse than the overall score would make it look.
One thing to note here is that, for the holdout benchmark cases where we don't include compile time as part of the benchmark and we repeatedly run searches, AOT FRE outperforms on the benchmark. For a lot of use cases, you don't want a regex that takes multiple seconds to compile, but there are plenty of cases where this is fine, e.g., for something like ripgrep or Silver Searcher, it could start running with a regex that can start matching right away and then compile in another thread and cut over to the faster matcher when it's done compiling. Given how much of my CPU is spent on long ripgrep searches, it seems like a strategy like that could improve performance for work I personally do. Before LLMs, it probably wouldn't have made sense to spend the effort to write an optimizing regex compiler, but this is now do-able with a few tokens.
Another thing to note here is that this comparison is arguably unfair because this was run on an ARM Graviton machine with SVE/SVE2 and FRE has SVE/SVE2 optimizations. Pre-LLM, it might not have been worth it to have regexes optimized for every combination of SIMD instructions out there, but with LLMs, it's fairly easy to generate ok-ish SIMD optimizations. I know human experts who find that they can generally outperform LLMs here, e.g., Jay Stelly said that the last time he tried getting an LLM to produce SIMD code, it took 20-some iterations to get the code as good as he wanted. But, on the flip side, LLMs have the capability to try more optimizations than a human could possibly try in any given amount of time, so they can still perform pretty well overall even if any specific optimization isn't as good as a human expert would produce.
There's also the problem discussed in this post of overfitting. Depending on the context, that problem is somewhere from very easy to solve to a bit difficult to solve. I deliberately didn't try very hard to solve the problem here to see what would happen, but I did manage to solve the problem without an outsized amount of effort when working on this Azul AI (just for example), but a lot of these big benchmark claims come when people spend little to no effort trying to avoid overfitting, or even negative effort. In the pre-LLM era, people would often pick highly unrepresentative microbenchmarks to show off how great their pet project is which, at least at a non-conscious level, involves negative effort to avoid overfitting to a benchmark. Due to how humans are, I don't think people are going to stop making misleading claims and it's become easier than ever to make misleading claims, so of course we see more of them.
Note that while this post has discussed non-AI software, everything said here goes double for AI software. For example, I've seen lots of people drop comments saying that Kimi K3 is Fable (5) level. But every single person I know who's used it has found it to be substantially worse than GPT-5.6 Sol and Fable. I'm not saying it's not an impressive engineering achievement, but the performance on a wide variety of real-world tasks isn't up to the level it is in benchmarks. This even applies to various eval-y problems, such as when a friend tried different coding agents on the ICFP 2026 contest problems. It also applies to security issues, which are something that I have no doubt AI labs are putting into their evals, e.g., a colleague of mine tried using Kimi K3 to scan for vulns in our software and found that it found approximately a quarter of the vulns GPT-5.6 Sol found, found no vulns that GPT-5.6 Sol didn't find, and didn't have any advantages in any dimension other than on cost. The people I know who are using cheaper models to find real security issues are using other models, such as GLM-5.2, which perform worse on benchmarks but better in practice.
Big 6-Week Airline Strike Wins a Major Pay Hike (1966)
SIXTY YEARS AGO, ON AUGUST 19, 1966, the International Association of Machinists won a hard-fought strike that had shut down more than sixty percent of the U.S. airline industry for six weeks during what otherwise would have been the airlines’ most profitable season.
The strike pitted 35,000 workers against some of the country’s biggest and most profitable airlines – Eastern, Northwest, National, TWA and United – four of which have since disappeared as a result of mergers.
The union was not only fighting the employers, they were also up against the U.S. government, which was trying to control inflation with a “voluntary” wage-increase cap of 3.2 percent a year. The airlines, which were enjoying unprecedented profits, took the government’s lead and offered the union only 3.2 percent, claiming it was their patriotic duty to abide by the voluntary guideline.
The union and its members insisted on substantially more, both because their frozen wages were then below the pay of many machinists in other industries and because the airlines were posting unprecedented profits.
The airlines refused, so on July 8, 1966, picketlines went up at 230 airports. Roughly 150,000 people with tickets to fly that day had to make alternative travel arrangements or stay home. Seats on planes flown by other airlines were almost non-existent because in 1966 airline schedules were tightly regulated, so the carriers that were still flying didn’t have the legal option to add flights to their existing schedules. Over the next six weeks, some 6 million would-be air travelers were diverted to buses, trains, or automobiles.
The White House did what it could by hosting contract negotiations, which managed to reach a tentative agreement calling for an annual 4.5 percent wage increase. But union members rejected the deal by a 3-1 margin.
Finally, when the strike was nearly six weeks old, the airlines offered the machinists pretty much what they asked for before the strike began, an annual six-percent wage increase with a cost-of-living adjustment. The membership approved the deal by a 2-1 margin and the U.S. airline industry’s most disruptive strike was history. https://youtu.be/RKhT1cbxOXE?si=dZNXbxlE_yHx6aRY
When the 8-Hour Workday Was a Radical Idea (1866)
ONE HUNDRED AND SIXTY YEARS AGO, ON AUGUST 20, 1866, the first session of the 5-day National Labor Convention began in Baltimore. Before the convention was over, the delegates agreed to form the first U.S. labor federation, known as the National Labor Union.
The convention issued the historically significant call to establish the 8-hour day and put an end to the era’s standard practice of 10- or 12-hour days. Calling for an 8-hour day in 1866 was considered a very radical position. When the brand-new National Labor Union did so, they beat Karl Marx’s brainchild. the First International Working Man’s Association, to the punch by two weeks.
In 1868 the National Labor Union played a central role in getting the U.S. Congress to mandate the 8-hour day for all U.S. government workers. When President Andrew Johnson vetoed the 8-hour law, Congress overrode the veto, putting the law into effect without the President’s signature. https://guides.loc.gov/this-month-in-business-history/august/national-labor-union-8-hour-work-day
Haiti Gives Slavery and Colonialism the Boot (1791)
TWO HUNDRED AND THIRTY-FIVE YEARS AGO, ON AUGUST 21, 1791, hundreds of enslaved people in northern Haiti began a planned uprising to reclaim their freedom and destroy the society that held them in bondage The insurrectionists were inspired not only by the hatred of slavery, but by the 2-year-old French Revolution’s endorsement of the Declaration of the Rights of Man and the Citizen. Haiti soon became the world’s first Black republic. https://portside.org/2020-09-03/black-spartacus-epic-life-toussaint-louverture
A Jury’s Verdict Makes Freedom a Reality (1781)
TWO HUNDRED AND FORTY-FIVE YEARS AGO, ON AUGUST 22, 1781, a court in Massachusetts ruled that Elizabeth Mumbet Freeman could not legally be another person’s property under the terms of the state’s 10-month-old Constitution, therefore the man who claimed to own her had no right to do so and she was, in fact, no longer enslaved. When the court ruled that the 37-year-old Freeman was free, she had been enslaved since she was born.
The Massachusetts Constitution said nothing about slavery and did not expressly outlaw it. But the Constitution’s first sentence reads “All men are born free and equal and have certain natural, essential, and unalienable rights; among which may be reckoned the right of enjoying and defending their lives and liberties; that of acquiring, possessing, and protecting property; in fine, that of seeking and obtaining their safety and happiness.”
When Freeman overheard the man who claimed to own her discussing the new constitution with dinner guests, she realized that she, and thousands of other Bay State residents who were being treated as property, could not be enslaved in the eyes of the law, She, and a fellow worker who was being treated as property by the same man, found a lawyer, Theodore Sedgewick, who filed a suit on their behalf, pointing out that the claim the two were enslaved had no legal basis. Three months later, a County Court of Common Pleas jury ruled that Freeman and her fellow worker were, in fact, free. In addition to their freedom, the court granted each plaintiff 30 shillings, the approximate equivalent $150 today.
The jury’s verdict only applied to the two plaintiffs, but two years later the Massachusetts Supreme Court ruled that under the Constitution’s Declaration of Rights no Bay Stater had a legal basis to claim ownership of another person.
Freeman had this to say about her experience: “Any time, any time while I was a slave, if one minute's freedom had been offered to me, and I had been told I must die at the end of that minute, I would have taken it—just to stand one minute on God's airth a free woman—I would.” https://constitutioncenter.org/blog/elizabeth-freeman-her-case-for-freedom-and-the-massachusetts-constitution
The Greenhouse Effect? Forget About It! (1856)
ONE HUNDRED AND SEVENTY YEARS AGO, ON AUGUST 23, 1856, Eunice Newton Foote presented her study Circumstances Affecting the Heat of the Sun's Rays at the annual meeting of the American Association for the Advancement of Science. Foote had discovered and demonstrated that sunlight heated carbon dioxide more than the other gases making up the atmosphere. Hence, the higher the atmosphere’s concentration of carbon dioxide, the hotter the atmosphere.
For reasons not known for certain, but reasonably ascribed to sexism, Foote’s discovery was forgotten for nearly a century. https://www.zinnedproject.org/news/tdih/eunice-newton-foote-confirms-greenhouse-effect/
An Ugly Moment in an Uglier War (1636)
THREE HUNDRED AND 90 YEARS AGO, ON AUGUST 24, 1636, one of America’s “founding fathers,” Massachusetts Bay Colony leader John Endecott, departed Boston at the head of 90-man war party, setting a course for Block Island, off the coast of Rhode Island.
It was the beginning of the Pequot War. Endecott’s orders were to kill all Native American men on the island and make prisoners of all women and children. When the punitive expedition reached Block Island, some 40 Native Americans attempted to prevent them from coming ashore, but were forced to retreat by Endecott’s guns.
Once ashore, the colonists encountered no more Native Americans. They spent two days on the 10-square-mile island, during which "they burned sixty native wigwams in the two villages they found and destroyed seven canoes and close to two hundred acres of native corn." https://www.thecrimson.com/column/fight-the-power/article/2023/11/13/williams-dename-winthrop-house/
Workers Lose a Big Fight on Blair Mountain (1921)
ONE HUNDRED AND FIVE YEARS AGO, ON AUGUST 25, 1921, the 9-day Battle of Blair Mountain, in southwestern West Virginia, started. It was one of the largest episodes of deadly violence ever to occur in the U.S., yet it is unknown to the vast majority of the U.S. population.
It is likely that the bloody battle is so obscure today because it is a perfect example of class warfare in action, pitting a racially mixed army of more than ten thousand union members and and supporters of the United Mine Workers of America against a smaller, but much better armed, force of anti-union cops, most of them employees of open-shop coal companies.
The dominant ideology of the U.S. pretends class warfare doesn’t exist, so a clear example of workers taking up arms in the name of their class is consigned to the memory hole.
The shooting ended – thanks to the intervention of the U.S. Army – in the defeat of the union supporters, almost all of whom gave up when the troops arrived, rather than fight against U.S. soldiers doing their legal duty. Many if not most of the union supporters had been members of the Army less than five years before during World War 1, and they would not fire on men wearing the uniform that had been theirs so recently.
Before the battle, small-scale, but deadly, class war had been raging in West Virginia coalfields for more than a year, during which dozens of union activists had been killed, one or two at a time, by anti-union thugs. The mobilization of more than ten thousand armed workers had been sparked when company thugs murdered two well-known pro-union law-enforcement officials in broad-daylight, to prevent them from testifying in a case about anti-union violence.
After the killing of the officials, pro-union coal miners began an attempt to overrun the nearby non-union stronghold of Mingo County, where they hoped to oust the local anti-union government that was behind much of the violence. Their attempted invasion of Mingo was prevented by the much better-armed anti-union force, which used well fortified positions on the slopes of Blair Mountain to block the workers’ advance.
Even though many tens of thousands of shots were fired during the fighting, most of the gunfire took place at very long range, so the number of people killed was probably less than twenty. The exact number is not known.
The victory of the coal barons and their thugs with the help of the U.S. Army proved to be a demonstration that anti-union violence could be committed with relative impunity, particularly in places where organized labor was not already powerful. The legal clout of anti-union bosses kept most unions and many of the members on the defensive until 1933, when the union-friendly National Industrial Recovery Act became law. https://www.nps.gov/articles/000/the-battle-of-blair-mountain.htm
For more People's History, visit
https://www.facebook.com/jonathan.bennett.7771/
This is a link post by Simon Willison, posted on 17th August 2026 .
ai 2,191 generative-ai 1,940 llms 1,907 qwen 60 ai-in-china 106 artificial-analysis 9
Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.
Pay me to send you less!
Sponsor & subscribe
Supporters cheer during an election night event for Minnesota Lt. Gov. Peggy Flanagan, a Democratic candidate for U.S. Senate, Tuesday, August 11, 2026, in Minneapolis. | Credit: Ellen Schmidt/AP Photo
It was a mixed bag for the left in Tuesday’s primary elections. In Wisconsin, DSA candidate Francesca Hong, who was far ahead in the polls and expected to win easily, instead lost in a squeaker to David Crowley. But in Minnesota, the state’s lefty lieutenant governor Peggy Flanagan easily whipped the moderate Rep. Angie Craig in the primary for that state’s U.S. Senate seat, by a nearly 20-point margin.
While the Wisconsin result was disappointing, if progressives had to pick, it was likely more important for Flanagan to win than Hong. In Wisconsin, what state Democrats can do will depend more on whether they can take control of the state legislature in the upcoming election and win total control of state government for the first time since 2010. Moreover, by all accounts Crowley is a fairly standard-issue liberal who would probably sign most progressive legislation; he has already announced plans to campaign alongside Hong. Craig, by contrast, is a loud centrist who voted for the Laken Riley Act , which granted sweeping powers to ICE—which would, of course, go on to mercilessly gun down two Minnesota residents—as well as for a resolution thanking the agency.
Whatever these races say about the state of the left—I would call it a story of gradual building success over time, with some unsurprising setbacks—the polls and turnout numbers have some more reliable lessons. They show that the Democratic base is boiling with energy and champing at the bit to vote against Republicans.
The polls for the Wisconsin race were off by roughly 20 points —about the same amount they were off in Minnesota (though in that case, they underestimated the progressive Flanagan ), and in the Michigan race last week. Clearly, something is rotten in the state of cellphone autodialers. Polling primaries was tough business even 30 years ago when everyone had a landline and calls from strangers were not 95 percent spam, because one has to gather data and then estimate how many people are actually going to turn out. That can vary wildly depending on the state, the party, whether it is a competitive race, the national environment, and so on.
Incidentally, the vaunted prediction markets also completely flubbed this one. They couldn’t even sort out who was likely to win when votes started coming in and the political pros could guess with increasing accuracy who was likely to win based on where the remaining votes were located. They were quite obviously just swinging wildly back and forth with the headlines. It turns out markets are not some infallible distillers of Truth from the celestial ether.
I’d guess that the pollsters are not getting good samples for these races, but they almost certainly missed the turnout. Let’s compare this year to 2018, the last time there was a contested Democratic primary in Wisconsin and an uncontested Republican one, to make it a fairly apples-to-apples comparison. Six years ago , about 539,000 people cast Democratic ballots, as compared to 455,000 Republicans. This year, with a few thousand votes still to count, it was more like 825,000 Democrats and 490,000 Republicans.
It’s a similar story in other recent primaries, where 2018 is once again the most recent similar year. In Minnesota in 2018 , about 584,000 Democrats and 320,000 Republicans turned out; this year, the numbers were 700,000 and 405,000, respectively. In Michigan in 2018 , about 1.1 million Democrats and 990,000 Republicans turned out; this year, the numbers were 1.5 million and 931,000, respectively.
The 2018 elections were nearly a clean Democratic sweep in these states. They won the U.S. Senate seats and the governorships in all three, and missed control of the Minnesota Senate by one vote. They would have won the Wisconsin State Assembly too if Republicans hadn’t rigged the district boundaries so egregiously that an eight-point defeat in the popular vote became a nearly two-thirds majority in seats. (Luckily, thanks to Democrats flipping the Wisconsin Supreme Court, the state legislative maps are now reasonably fair.)
Primary turnouts are a decent, albeit imperfect, historical proxy for general-election performances. And if there is any major difference between today and 2018, it’s that Donald Trump, with his lunatic tariff obsession and unwinnable war in Iran, is substantially less popular than he was back then. Unless something big changes, Republicans are liable to be looking at a generational wipeout in November.
Ryan Cooper is a senior editor at The American Prospect , and author of How Are You Going to Pay for That?: Smart Answers to the Dumbest Question in Politics . He was previously a national correspondent for The Week . His work has also appeared in The Nation , The New Republic , and Current Affairs.
AOC shares her egg-freezing journey online | screen grab
The latest shot heard ’round the world came not from a gun but from a syringe of fertility hormones belonging to Representative Alexandria Ocasio-Cortez.
Last weekend on Instagram, the congresswoman announced her plan to freeze her eggs and document the process. Ms. Ocasio-Cortez, seated on a sofa, showed the array of medical supplies that she will need to prepare her body for an egg retrieval. Cautioning viewers not to “be weird about this,” she pulled up her white top, swabbed her bare abdomen with alcohol, took a deep breath and injected a fold of flesh she pinched with her fingers. “That wasn’t so bad,” she told us with visible relief.
This was a revolutionary moment.
It’s not at all unusual that Ms. Ocasio-Cortez, a 36-year-old professional woman, might make this choice. And it’s not surprising that a politician as media savvy as she is might talk about that choice online. Last year Representative Sara Jacobs did so , too, documenting her own egg-freezing experience.
But heavens, seeing a public figure in such a vulnerable, private moment — literally baring her underbelly — still carries quite a jolt. And the reality of Ms. Ocasio-Cortez’s huge public following only intensifies this effect. By now, millions of people surely have seen her egg-freezing videos.
Many applauded her transparency and bravery. Others insisted that her private life should be kept private. And some right-wing commentators seized on the occasion to proclaim their pronatalist bona fides. The conservative blogger Matt Walsh lashed out on X, for example, calling Ms. Ocasio-Cortez’s decision “baffling” and boasting that he would have kids in their 20s “by the time I turn 50,” dismissing later-in-life pregnancy as “backwards and ridiculous.”
By documenting her egg-freezing process, Ms. Ocasio-Cortez showed herself seizing control of her reproductive life while leaving the topic of marriage out of the equation. She has made her body public but offered no comment on her domestic situation — a reversal of the usual order.
But she is doing more than that. Ms. Ocasio-Cortez has ushered in a new era — call it the new age of young female leadership. And she is doing it in conjunction with an oblique suggestion on the ABC Sunday show “This Week” about a 2028 White House run. (She said she hasn’t “ruled out the possibility.”) On Sunday Ms. Ocasio-Cortez documented herself administering another hormone injection in the ABC green room. (She brought her supplies to the TV studio in a lunch box.) This took place against a backdrop of an enormous picture of the Capitol. The connection could not be any clearer.
Ms. Ocasio-Cortez is getting ahead of all the skeptical questions she rightly anticipates regarding her suitability as a candidate for the highest office, such as: How could we let a childless woman in her late 30s be president? Isn’t she likely to get pregnant during her first term, when she’d be between the ages of 38 and 42? And given that, wouldn’t she be unable to perform her political duties? By attempting to postpone motherhood, Ms. Ocasio-Cortez is telling us not to worry; she’s thought of everything. She grasps the concern and is addressing it. Never mind that many men who have recently run for president have not provided answers to questions about whether their aging bodies will leave them fit for leadership throughout their term.
Like millions of other women her age, Ms. Ocasio-Cortez is grappling with the impossible goal and financial burden of having it all. To the women negotiating that tight window of time between their 30s and early 40s, when they want to build both careers and families, she is saying: I am one of you. Feminism has done little to make this balancing act any easier. Despite all the rosy claims to the contrary, the burdens of family still fall disproportionately on women.
I’d been to Music City Center in Nashville before, but this day was different. I sat with colleagues and ran into other people I knew from the progressive movement, but we had not convened to discuss organizing strategies or hear from a candidate. We were there to mourn the loss of a grand force in not only Nashville, but all of Southern labor: Vonda McDaniel.
McDaniel’s presence loomed large all throughout the center’s 2.1 million square feet, literally and figuratively. Photos of her adorned the monitors across the building, and she had served as vice-chair of the Music City Center board, ensuring that organized labor had a seat at the table of Nashville’s largest convention center. Charles Starks, president of Music City Center, called it “the house that Vonda built.”
Hundreds of people had gathered from across the country on July 8 to pay their respects to McDaniel, who at the time of her death was transitioning from her role as president of the Nashville and Middle Tennessee Central Labor Council (CLC) to co-executive director of the prestigious Highlander Center.
McDaniel, a Nashville native, worked at the Bridgestone Firestone plant where she joined United Steelworkers (USW) Local 1055. She rose in union leadership and became president of the Nashville CLC in 2013. In this role, she brought together strong coalitions of labor, social justice organizations, and faith groups like Stand up Nashville, Nashville Justice League, and Tennessee for All. She also served on the national AFL-CIO Executive Council.
And for me, Vonda McDaniel opened the door to the labor movement and all it could be. I was a young activist in Nashville preparing for a direct action when someone told me I should reach out to the Nashville CLC. I didn’t understand. What did unions have to do with social justice?
Seeing Vonda bring people together from all walks of life inspired me. My typical organizing circles were full of people who went to the same schools and shared the exact same politics and held the same types of jobs. At one of the CLC’s monthly luncheons, you could find teachers talking with ironworkers and politicians learning from electricians—all races, genders, and educational backgrounds represented. This was the kind of movement that social justice activists like me said we wanted to build, but it already existed, and the CLC that Vonda McDaniel steered for more than a decade gave us a living example.
Central labor councils have been a critical piece of infrastructure for the American labor movement as far back as the end of the 19th century. With the merger of the American Federation of Labor and the Congress of Industrial Organizations in 1955, central labor councils and area labor federations became the go-to hubs for local organizing.
Where national unions organize occupationally, central area labor councils organize geographically, becoming a logical home for the work of advancing labor’s needs politically at a local level. Labor councils are like a union of unions. Union locals represent workers at one business or in one bargaining unit in a large firm. They are branches of national unions, and join labor councils as affiliate members. For example, International Association of Machinists and Aerospace Workers (IAMAW) is affiliated with the national AFL-CIO and represents workers across the United States and Canada, while IAMAW 44 represents workers at United Launch Alliance in Decatur, Alabama and is an affiliated member of the North Alabama Area Labor Council.
Labor councils are categorized by the IRS as a 501(c)(5). This allows them to accept (non-tax-deductible) donations and do political work like endorsing candidates.
But like all limbs of the labor movement, labor councils hit a slump as the 20th century progressed into the 21st. In the U.S. union density reached a peak of 33.5% in 1954 , but had dwindled to a record low of 9.9% in 2024 ; in Alabama the descent was 21.1% in 1964 to 6.6% in 2024 .
After I left Nashville, I returned to my hometown of Huntsville, Alabama. I found a position at a great nonprofit that voluntarily recognized our union, represented by CWA 3908, instead of forcing us to have a tedious and drawn-out election process. My first task as a union member was to join the local labor council. I’m now vice-president and doing my best to continue Vonda’s legacy of Black women labor leaders.
Huntsville is certainly not what one immediately thinks of as a union stronghold, past or present. But manufacturing played a significant role in North Alabama through the beginning of the 21st century. Then United Rubber Workers Local 915 took a big hit when the Dunlop plant closed in 2003. Lawrence County’s largest employer, International Paper, represented by Steelworkers Local 1137, started layoffs in 2000 before finally shutting down in 2014.
Morrison sees a successful labor council as acting as “representatives of the working people.” That representation can come by electing pro-labor candidates and advocating once they’re in office, supporting organizing drives, and educating the general public about the labor movement.
The aerospace industry that earned Huntsville its nickname of Rocket City continues to thrive due to the large federal presence that developed space exploration technology in the 1950s and ‘60s. More than 20,000 aerospace professionals call Huntsville home today; Redstone Arsenal, home to the Marshall Space Flight Center and more than 70 other organizations, is the city’s largest employer. And many of its employees are union members.
The American Federation of Government Employees (AFGE) is a labor union that represents around 10,000 workers on Redstone Arsenal, from firefighters to systems analysts. The International Federation of Professional and Technical Engineers (IFTPE) represents approximately 1,400 NASA employees in Huntsville.
After an internship at the Army Corps of Engineers, Jacob Morrison started as a project manager in 2018 and joined AFGE Local 1858. He quickly rose up the ranks at his union, becoming an Assistant Vice President (essentially a shop steward) by the time he was 25. After organizing on the University of Alabama at Huntsville campus for Sen. Bernie Sanders’s presidential campaign in 2016, he had gotten discouraged about the potential impact of electoral politics alone. The labor movement offered people a way to make their lives better through collective action, in their workplaces and in the community more broadly. But he saw a gap.
Huntsville still had dozens of active union locals with thousands of workers represented, but they didn’t have a collective voice. To remedy that, he sought to revitalize the North Alabama Area Labor Council (NAALC), the umbrella group for unions in the area. Morrison sees a successful labor council as acting as “representatives of the working people.” That representation can come by electing pro-labor candidates and advocating once they’re in office, supporting organizing drives, and educating the general public about the labor movement.
Newspaper stories from the last half of the 20th century highlighted the NAALC’s role in North Alabama politics: endorsements, statements on the day’s news stories, strike support, and more. But by 2018 when Morrison started asking how he could get involved, he said, ”I was told that I just missed it. Literally. It was de-chartered in 2017.”
He jumped in, becoming the youngest labor council president in the state. He and his colleague turned vice-president, David Story (Machinists Union Local 44) started with five affiliated locals, the minimum required by the AFL-CIO to charter a CLC. They applied for the charter, received $3,000 from the previous labor council’s bank account and put it in the Bank of Labor.
“I would like to get every local union in the area affiliated,” he says. “I’d like to see us be a counter to the Chamber of Commerce.”
Morrison made sure entry to the labor council was accessible. He set the “per caps” at $.10 per member per month, meaning that if an affiliated local had 200 members, they were charged $20/month or $240/year for dues.
The resurrected NAALC held its first meeting in 2019, and was fully rechartered in April 2020, eventually growing its membership to 12 affiliate local unions representing thousands of workers across seven counties in North Alabama. These workers include machinists, letter carriers, and stagehands.
In organizing, Morrison still finds that the first barrier is often demonstrating the value of the NAALC to local unions.
“Very few people want to get to a party before it starts,” he said of getting new members and locals engaged. His role is also unpaid, so he has limited time and resources to dedicate.
Still, Morrison and the other members of the NAALC are proud of their accomplishments. They’ve built relationships with local press, and with the support of AFL-CIO staff, have achieved consistent and favorable media coverage. In Summer 2024, the NAALC sent out candidate questionnaires and gave out its first endorsements and canvassed to help elect a retired union member who campaigned on working-class representation to become Huntsville’s first Black woman city councilperson.
This spring, the NAALC’s Community Solidarity Committee collaborated with local advocates and community members to show resistance to a hospital consolidation that effectively created a healthcare monopoly for half the state; they held a listening session, released a report, and met with hospital leadership.
And Morrison still has big goals for the council.
“I would like to get every local union in the area affiliated,” he says. “I’d like to see us be a counter to the Chamber of Commerce.”
In his work as the NAALC president as well as host of The Valley Labor Report , a union radio show that airs in five states, Morrison has met and learned from other labor leaders across the country.
One of those leaders is Tevita Uhatafe, president of the newly formed North Texas Area Labor Federation (similar to labor councils, area labor federations are regional labor bodies). Formerly president of the Tarrant County Labor Council, Uhatafe says support from state leadership is essential for growing a labor council or area labor federation.
Uhatafe did not grow up in a union family. Instead, his Tongan community grounded him in the elements he would come to recognize in the labor movement: solidarity, service, and loyalty. When he started working at the airport he joined Transport Workers Union Local 567. Affable by nature, he found the Tarrant County Labor Council was eager for new members and young leaders.
Now the president, Uhatafe continues to develop young leaders through YALL. The Texas AFL-CIO runs Young Active Labor Leaders or YALL , which provides a space for young adults on all labor council boards throughout the state. Uhatafe sees this as an essential tool for ensuring the labor movement endures.
Uhatafe uses traditional and social media to build credibility and support. He also engages with community members on a variety of projects. One of these is a school supply drive with teachers. This not only brings the community together, but helps facilitate organizing conversations with teachers that can often be difficult to start.
Elsewhere around the country, just in the last two years the Chattanooga Area Labor Council supported a successful UAW drive at Volkswagen, and the Pierce County Central Labor Council secured $5 million from the state of Washington to create a childcare and workforce hub.
Labor councils act as a breeding ground for leadership: Beau Hawk, formerly of the Knoxville-Oak Ridge Central Labor Council, had a prominent campaign for Knox County Mayor and closed the gap in a predominantly red county and before her death Vonda McDaniel had been tapped to serve as the co-executive director of the prestigious Highlander Center.
Labor councils can also pilot big changes in direction, though these are not always welcome or sustainable. In 2019, the Vermont State Labor Council (VSLC) elected new leadership to their executive board: The Vermont AFL-CIO United! slate was a group of progressive rank-and-file unionists who were excited to rebuild the VSLC’s organizing and advocacy strategies.
“We were running worker circles while we were in office, where people could come, organized or unorganized, and share wisdom, information, tools, strategies,” Ellen Kaye, VSLC’s former executive vice-president said.
They made bold moves like using regular workers as policy advocates instead of paid lobbyists, supporting Green New Deal legislation, opening up meetings to all union members not just paid officers, and endorsing third party candidates. “The basic principle of organizing is calling somebody up and saying, ‘Hey, I’d love to hear your thoughts on this thing,” Kaye said.
Their visionary approach paid off in legislative and organizing wins, and they started to receive attention for these changes. These wins and changes brought a high level of visibility to the small state.
In 2020, the Vermont State Labor Council passed a resolution to call a general strike if Trump refused to leave office, in direct violation of a decision by the Executive Council of the AFL-CIO. The national president at the time, the late Richard Trumka, condemned the resolution, investigated VSLC president David Van Deusen for misconduct, and threatened to put the VSLC into a trusteeship as he felt the resolution conflicted with the national AFL-CIO’s electoral strategy.
After Trumka passed away in August 2021, the new AFL-CIO president Liz Shuler continued to investigate the VSLC. The United! slate won in 2023, electing Katie Maurice as the youngest state federation president in the country (and the only DSA member at the time). The national AFL-CIO nullified these election results, ordering a new election with new rules in May 2024. Larry Moquin, who had run and lost against Maurice in 2023, was elected in that contest, andreelected in September 2025. Currently the executive board of the VSLC is a mix of United! supporters and others.
“We had people but we weren’t organized enough,” Kaye said. “I would say that’s my word of caution to people. If you’re going to do a progressive slate and really try to accomplish things, people have to be committed.” The VSLC experiment offers lessons to other labor movement visionaries who challenge established leadership.
Whether serving as sites of innovation or incubators of solidarity, labor councils are formidable, although sometimes overlooked, institutions within the labor movement. They’re an essential organizing tool and a way for unions to act collectively.
[Whitney Washington is a writer and advocate based in Huntsville, Alabama. She is a longtime advocate, having worked for political campaigns and progressive advocacy organizations in the South for more than a decade.
She is a regular contributor to Burnaway, a magazine of contemporary art and criticism from the American South and the Caribbean, since 2024. She is an alum of the Black Embodiments Studio’s Arts Writing Incubator, Tin House Summer Workshop, and Kenyon Review Writers Workshop; she was also a 2022 Periplus fellow. She has been published in Electric Literature, Oxford American, and Roxane Gay’s newsletter The Audacity.
Whitney is active in Alabama’s labor movement. She is the Vice President of the North Alabama Area Labor Council and a member CWA 3908.]
St. John’s, Newfoundland and Labrador
In a more dangerous and divided world, Canada’s new government is focused on what we can control: building a stronger, more independent, and more sustainable country.
Canada is starting from a position of strength. We have what the world wants. We have the natural resources, critical minerals, and clean power that will build this century. We have the workers and the ingenuity to turn that potential into strength. We understand we can build faster and bigger when we work together. And we have trust, the most valuable commodity in an increasingly volatile, unreliable world.
We have everything we need to build the future we want.
Today, the Prime Minister, Mark Carney, alongside the Premier of Québec, Christine Fréchette, the Premier of Newfoundland and Labrador, Tony Wakeham, and the CEOs of Hydro-Québec and Newfoundland and Labrador Hydro, announced a historic agreement to upgrade and expand the Churchill Falls Generating Station and develop the Gull Island project and related transmission, building one of the largest electricity projects in North America.
As part of this agreement, the federal government will provide $10 billion in federal financing to:
Together, these projects represent the largest clean energy investment in North American history, at nearly $70 billion. They will generate 14,000 megawatts of clean, renewable power – nearly tripling the current generating capacity of Churchill Falls. That is enough power to light, heat, and cool all the homes in Toronto, Montréal, and Vancouver combined. These projects will support 23,000 jobs – from the skilled trades to engineering – and contribute $31 billion to Canada’s GDP through the early 2040s.
The Labrador Trough is a world-class mining region across Newfoundland and Labrador and Québec, with significant sources of high-purity iron ore. To harness this strategic asset and create high-paying careers for Canadian workers, the government announced the referral of the Labrador Trough Clean Power, Critical Minerals and Infrastructure Corridor to the Major Projects Office (MPO). The MPO will coordinate and structure federal financing, accelerate permitting requirements, and work with Indigenous Peoples to forge meaningful partnerships.
In addition, Canada’s new government will support strategic, pre-development projects in the Labrador region , funded under the First and Last Mile Fund (FLMF) through Natural Resources Canada. This includes the following projects:
We are making Canada the best place in the world to build and invest in clean energy. By combining the Clean Economy Investment Tax Credits, efficient project approvals, cooperation agreements with provinces and territories, strategic investments, and attractive financing, we are giving builders the certainty to build big and move fast.
Today’s agreement realises the enormous potential of Canada when we build together.
This is cooperative federalism at work.
In a more dangerous and volatile world, we’re choosing to build clean, affordable, and reliable energy systems, in partnership with all Canadians, for all Canadians.
“Canada is extending its unique advantage in clean, reliable, and affordable power. Because when we master energy, we master our destiny. Through cooperative federalism, we are unlocking our immense potential, building big, building sustainably, building in partnership, and building Canada strong for all.”
“In the current geopolitical context, it is vital for every nation to secure its energy future. I am very proud to announce today the conclusion of this new agreement, which is a priority for Québec, particularly for generations to come. Through this partnership, we are supporting the energy transition and enabling the growth of our economy through renewable energy, while ensuring our energy independence.”
“This is a win-win-win for Newfoundland and Labrador, the federal government, and Québec. I want to thank Prime Minister Carney and Premier Fréchette for their leadership throughout this process. We are finally replacing the notorious 1969 Churchill Falls deal and the 2024 MOU with a new deal that will guarantee us more power, more value, and more transmission. Newfoundlanders and Labradorians will finally be the primary beneficiary of our own resources, with complete control over whether we use our power to develop our economy or sell to outside markets. Today is not about what we can tear up, it’s about what we can build up. It is now time for us to roll up our sleeves and get to work to build a better and brighter future for all of us.”
Behalve in Nederland zijn er ook Repair Cafés in België, Duitsland, Frankrijk, het Verenigd Koninkrijk, de Verenigde Staten en in tientallen andere landen verspreid over de hele wereld. Het Repair Café is zelfs doorgedrongen tot India en Japan!
0
Schatting aantal betrokken vrijwilligers
0
Schatting aantal gerepareerde voorwerpen per maand
Repair Café International ondersteunt lokale groepen in de hele wereld bij het opstarten van hun eigen Repair Café. Voor een eenmalige vrijwillige bijdrage bieden wij een digitaal startpakket.
Hildegard in her new gear
While much of the country was staying put during COVID, I moved to the SF Bay Area and adopted Hildegard. The Bay Area is a paradise for dogs, but what I didn’t know is that a silent danger lurks in the grass [cue dramatic music] the foxtail weed!
Foxtails are a pernicious seed that find their way into dog’s ears, nose, eyes, and skin. They have unidirectional barbs that cause them to quickly migrate into the dog, causing infection and even death. The majority of dog owners seem to think that the risk is low enough to chance it. We got unlucky - Hilde got a foxtail in her ear two weeks into the season, which meant a trip to the emergency vet and negative 400 dollars.
The Outfox, looking like a cross between a dog muzzle and a piece of lawn furniture
Our first response was get an “OutFox Field Guard”, which is a $50 vinyl coated mesh bag that covers and protects the dog’s head. I noticed the Outfox had a few problems:
If you set aside the lovely universality and simplicity of a bag, I thought it might be possible to make foxtail protection that fits better, is more durable, and doesn’t block her mouth for treats and fetch.
One initial idea was to modify the existing bag with a port at the top, probably similar to a rubber wire grommet. I didn’t go this route, though I might still try it, because it doesn’t solve the durability problem. Another idea was to reinforce the mouth area with some other, stronger mesh. Didn’t do this, because it’s boring and probably would make it even more difficult for her to carry anything in her mouth (which depends on the mesh flexing enough to tuck into her mouth around a ball/stick/whatever).
The design I built replaces the soft bag with a thin helmet shell that conforms to the shape of her face. The shell would have replaceable mesh inserts for the nose and ears, to keep it breathable and avoid blocking her hearing or sense of smell.
Dogs are hard to measure. They’re all curves, they are fuzzy instead of solid, and they move around a lot. My “good” 3D scanner (an HP SLS 3) wasn’t going to work. The scan speed is way too slow, and it doesn’t do well with hair or fur. The only other option available to me is the iPhone Truedepth sensor, which as it turns out is a fairly acceptable scanner when you use the right app. I used Scandy Pro. If you want to scan your animal, it works pretty well. For the higher resolution scan, I needed to use the front camera, which makes seeing what you’re doing rather challenging. Scandy sells a little periscope mirror to flip the scanner so you can point it forwards, but I was impatient and got a good enough result without it.
Looks rough, but enough of her head shape is there to guess at the rest.
I’m going to come clean here. My design is clearly derivative of a piece of existing equipment I’ve seen on police dogs. Using this concept, I jumped directly into CAD, without spending too many cycles wondering if it would succeed in attaching to Hilde’s head.
The steps to model an object like this were:
Once you have a shape that fits your intent, we switch to thinking with solids. The shell is given a uniform thickness (Solidify), and I draw volumes representing bolt holes and other features that are Boolean subtracted from the model. I considered using solid CAD (ie Onshape or Solidworks), but I found making 3D curved shapes in solid CAD software to be needlessly slow for my purposes here. Want to make a single 3D spline? Better be prepared to draw it twice from two angles, and don’t forget that every single control point on one curve has to correspond to the other! No thanks!
The final part:
I decided to make custom lenses, rather than design around off-the-shelf ones. I was curious to try a new lensmaking technique, where the outer (in this case 3D-curved) rim of the lens is fixed in place while heating a sheet of plastic, and the center part (which needs to be clear) is allowed to droop freely into a hole in the mold form.
Vacuum forming (where the lens is sucked against an explicitly defined shape) might have worked as well - but in my experience you get the best lens clarity when nothing touches the hot plastic during forming. A previous attempt at vacuum forming a motorcycle helmet face shield resulted in a hazed surface an a lot of layer lines where I needed clarity. It might be possible to flame-polish the haze away, I’ll have to try that route in a future build.
Here’s one of the lens molds in the vacuum former. There’s no temperature control, so I just heated the acrylic as far away from the element as possible:
After adding straps, here’s a view of the inside:
Left:
Design & fit prototype 1, printed in Draft resin.
Right:
Finished piece, with EVA foam padding and resin inserts to hold replaceable lenses and ear protectors in place.
The good: Quality-wise, I’m quite happy with the build. Since I wanted to get the outer shell to the printer ASAP, before I had finished designing the retaining inserts for the eye and ear protectors, I didn’t have many screw holes defined in the shell. To get things moving a little faster, I hand-drilled the holes for heat-set inserts using the retaining inserts as a drill guide - which worked like a charm, if a little labor intensive. Printing or molding the insert holes would be the way to go if this were manufactured at any scale.
Fabrication of the foam padding was a bit of an afterthought, because it’s normally hidden. It’s rather crudely hand-cut. If I had a laser engraver on hand, I’d have preferred to design and cut the pattern as a single piece.
The bad: In testing, my ignorance of the vagaries of dog psychology became clear. It seems like other dogs don’t react very well to this headgear, either. Larger dogs in particular find it threatening, which makes me cautious of using it at the dog park for Hilde’s safety. So instead, it will be reserved for hikes and other activities.
Several people have told me that dogs are very sensitive to silhouettes. I had hoped that the silhouette of this helmet would be closer to “dog” than the OutFox, but evidently that’s not the whole picture. I might try making a v2 where both ears are in a single “bonnet”, perhaps that would be different enough to be unusual rather than threatening. Other questions for the field of dog psychology: Does color play a role? Maybe painting with dog-like colors would help.
The silver lining: The helmet seems comfortable on Hilde, and it works very well as a mounting point for a GoPro. It’s very fun to see the world from her perspective. Who knew dogs move so fast?
Let me know if you have other ideas for how to make a better protective helmet, or other interesting gear I can make for Hildegard Von Bingen!
“High Arabesque” , and other works inside.
When Georgia Attorney General Chris Carr wrote an opinion piece on the importance of police surveillance technology like license plate readers, he included an entire section under the heading, “How LPRs have helped law enforcement solve crimes.”
But does the vast network of surveillance equipment really help solve crimes? An in-depth review of FBI data across a range of crime types suggests that this infrastructure has not corresponded with an increase in the Atlanta Police Department’s (APD) clearance rates.
Since Flock Safety’s launch in 2017, its black-boxed license plate reader cameras on black poles with a solar panel atop have become ubiquitous across the country, including here in Atlanta, where the company is headquartered. Flock’s software platform boasts a nationwide lookup feature, allowing police officers to search over 80,000 cameras—both license plate readers and traditional cameras—with the click of a few buttons.
Increased scrutiny over Flock cameras arose in May 2025, when 404 Media revealed the nationwide lookup feature had been repeatedly used to search for immigration enforcement. Locally, ACPC found evidence that APD also conducted immigration-related searches in the months following President Donald Trump’s second inauguration. The department denies the assertion.
These cameras, surveillance technology officials and policy makers say, play a vital role in solving crime. In 2024, Flock claimed that “one additional Flock Safety License Plate Recognition (LPR) camera per sworn officer correlates with a 9.1% increase in clearance rate,” and that “20 additional Flock customers within 50 kilometers of the original agency leads to a 1% increase in clearance rates.”
According to the FBI , an offense is cleared by arrest or solved for crime reporting purposes. Police can also clear cases by exceptional means, including the identified alleged offender’s death, a victim refusing to cooperate or prosecutors declining to pursue the case. The clearance data was pulled from the FBI’s Crime Data Explorer , a tool, the bureau says is intended to increase police accountability by providing greater access to law enforcement data. The clearance numbers in this story are based on the quantity of reported crimes and clearances. Clearances in this data set do not necessarily happen in the same month or year as the underlying reported offense.
The company and APD have both been silent on the exact number of Flock cameras in Metro Atlanta; however, DeFlock.org , which has a crowdsourced map of surveillance cameras, shows over 5,000 Flock cameras in the area. APD has an authorized strength of 2,000 sworn officers, though it currently employs around 1,800.
Despite the growth of the Flock network, APD’s clearance rates have not increased markedly between 2021 and 2025, the first and last full years for which comparable clearance data are available.
| Category | 2021 | 2025 | Difference | Highest Year | Lowest Year | Variation |
| Criminal Homicide | 53.4% | 48.0% | -5.4% | 61.9% | 48.0% | 13.9% |
| Rape | 37.7% | 37.7% | -0.0% | 37.7% | 21.2% | 16.5% |
| Robbery | 25.5% | 25.4% | -0.1% | 28.8% | 25.4% | 3.4% |
| Aggravated Assault | 37.6% | 37.8% | 0.2% | 39.6% | 34.5% | 5.1% |
| Burglary | 10.7% | 11.6% | 0.9% | 12.3% | 10.6% | 1.7% |
| All Other Larceny | 9.0% | 7.0% | -2.0% | 9.0% | 4.1% | 4.8% |
| Motor Vehicle Theft | 9.9% | 10.2% | 0.3% | 10.2% | 3.9% | 6.3% |
| Theft From Motor Vehicle | 1.4% | 2.8% | 1.4% | 2.8% | 1.4% | 1.4% |
| Shoplifting | 38.8% | 37.6% | -1.2% | 38.9% | 34.4% | 4.5% |
Flock is one of the two main arms of the city’s growing police surveillance infrastructure.
Fusus -connected cameras have also proliferated throughout the city of Atlanta over the past six years, largely thanks to the efforts of the Atlanta Police Foundation (APF) and APD. Axon , makers of the taser and body cameras, purchased Fusus in 2024. Fusus software allows for the integration of surveillance devices from different manufacturers onto a single platform.
In March, APD posted on social media, encouraging residents to add their doorbell cameras to the city’s camera network program, Connect Atlanta, which is jointly operated by the APF and the APD. “Did you know your home 🏡security and doorbell cameras could play a key role in solving crimes in your neighborhood❓,” the post said.
APD has reported 34 cases of human trafficking since October 2020. In the same time frame, the department has cleared seven cases. The most recent clearance was reported in the first quarter of 2024. Since then, an additional 12 human trafficking incidents have been reported by APD with no clearances.
Connect Atlanta runs on Axon-Fusus software and comprises two types of participation. The first, registered cameras , allow officers to know where cameras are and request footage from camera owners, but do not allow direct access. The second, integrated cameras , allow APD officers to view live streams or recordings—if the owner has the setting enabled—on demand.
Despite the eightfold growth of the integrated camera network between 2021 and 2026, APD’s clearance rate across eight major crime categories has remained relatively unchanged.
In November 2021, Atlanta Magazine reported that Connect Atlanta had 3,300 integrated cameras. That number grew to 10,188 by December 2022, according to an APD presentation to the Public Safety and Legal Administration (PSLA) Committee. Since 2022, APD has included an update on the number of cameras in its quarterly report to the committee. On July 23, 2026, Connect Atlanta’s website reported 28,626 integrated and 17,314 registered cameras.
For Atlanta City Council Member Kelsea Bond, the city’s growing surveillance network merits a deeper examination.
“Atlanta is the most surveilled city in the country, and even in the last several years, usage of security cameras and surveillance technology in our city has continued to explode,” Bond told ACPC. “The swift expansion of surveillance in our city raises serious concerns about civil liberties, discrimination and bias, and privacy and transparency surrounding how the data collected is used.”
These surveillance cameras are said to play a key role in solving major crimes. Even news articles critical of Flock Safety contain statements like, “ Flock Safety cameras help solve crimes .”
Murder, or homicide, is a frequently touted crime that cameras are alleged to help solve.
On Feb. 26, 2026, Flock CEO Garett Langley wrote on X, “Just spoke to an Atlanta PD detective who solved 35 homicides last year using Flock.”
APF CEO Dave Wilkinson spoke to his belief in the importance of these cameras during an Atlanta City Council Community Development and Human Services Committee meeting on May 18. “The murders—the major homicide cases in our city—are mostly all solved through the camera network,” Wilkinson said.
In 2021, APD had a 53.4% homicide clearance rate. In 2025, the department had a 48% clearance rate—its lowest homicide clearance rate between 2021 and 2025.
The department’s best year was 2023, when it hit a 61.9% clearance rate. Across the entire timeframe, APD had a 53.8% clearance rate.
Contrary to these findings, on Jan. 20, 2026, Atlanta Police Chief Darin Schierbaum held a press conference saying the department had reached a 77.55% homicide clearance rate for 2025.
APD did not respond to an opportunity to comment on the findings of this story or the discrepancy between the 77.55% homicide clearance rate claimed by the department and the 48% clearance rate in the FBI’s data, which is derived from agency reports to the bureau.
It’s not just homicide. Policymakers say these cameras help solve many types of crime.
When Bond tried to send a proposed Flock Safety camera contract renewal for Hartsfield-Jackson International Airport back to committee during the June 15 regular meeting of the Atlanta City Council, Council member Jason Dozier pushed back, saying, “I do have deep concerns about some of the challenges that we experience at the airport, particularly around human trafficking.”
Again, the data tells a different story. More cameras have not resulted in a greater percentage of human trafficking cases cleared.
APD has reported 37 cases of human trafficking since October 2020. In the same time frame, the department has cleared seven cases. The most recent clearance happened in May 2024 for an incident reported on March 12, 2024. Since then, an additional 12 human trafficking incidents have been reported by APD with no clearances.
“This data regarding the human trafficking cases is really interesting, since both that and car theft were used by certain Council members to justify renewing the airport Flock contract that I tried to send back to committee at a recent City Council meeting,” Bond said. “If our city is going to be spending tens of millions of dollars a year on surveillance technology, the least we can do is investigate whether this technology is even effective at solving cases.”
To have a more complete understanding of APD’s clearance rates, ACPC surveyed the data for nine crime categories: homicide, rape, aggravated assault, robbery, burglary, motor vehicle theft, theft from motor vehicle, shoplifting and all other larceny—a composite category of larcenies except shoplifting and motor vehicle thefts, which are tracked separately.
These are the categories APD presents in its crime statistics regular report to the Public Safety Committee . In these reports, the department shows how often these crimes occur, but not how often they are cleared.
“Those who seek to justify the City of Atlanta’s increased reliance on such invasive technology, such as Flock cameras, often fearmonger about crime, but that fear-mongering begs the question—does increased surveillance actually correlate with increased rates of solved crimes?” Bond said.
With the prevalence of tools like Flock license plate readers, it would stand to reason that more motor vehicle thefts are solved now than ever before. However, the data does not show a strong correlation.
In 2021, APD had a 9.9% clearance rate for motor vehicle thefts. Clearance rates dropped to 4.9% in 2023 and 3.9% in 2024 during a steep rise in motor vehicle thefts linked to the Kia vulnerability . The department rose back to a 10.2% clearance rate in 2025, amid an overall decrease in motor vehicle thefts—a trend seen throughout the county, according to the National Insurance Crime Bureau . Overall, between the fourth quarter of 2020 and the second quarter of 2026, the Crime Data Explorer shows 19,841 reported motor vehicle thefts and 1382 clearances, a 7% clearance rate.
So-called property crimes like theft from motor vehicles, shoplifting or other types of larceny also do not exhibit strong increases in clearance rates.
Thefts from motor vehicles are the most commonly reported offense type and the department’s least cleared among the nine categories, though its clearance rate has doubled, from 1.4% in 2021 to 2.8% in 2025.
There is one category of crime APD showed a marked improvement in clearing: drug crimes. The department reported a 65.8% clearance rate in 2021. That figure has grown steadily, hitting 90.1% over the first six months of 2026.
Cities and counties across the country have been canceling or pausing their contracts with Flock over privacy and security concerns with the company itself. DeFlock has found 91 such municipalities. But Flock, for many agencies, including Atlanta, is just one piece of surveillance infrastructure. DeFlock’s website also shows the crowd-sourced geolocations of other automated license plate readers beyond Flock.
For Len Phillips of DeFlock Atlanta, the question is why local governments continue to spend tax dollars on surveillance that hasn’t produced meaningful outcomes.
“The public has been asked to accept a significant expansion of surveillance based on the promise that it would dramatically improve public safety,” Phillips told ACPC.
Instead of improving crime-solving, public dollars have been spent on surveillance infrastructure that has caused more harm, Phillips said. “We’ve seen cases of abuse by police officers ,” he continued. “We’ve seen cases of faulty plate reads leading to felony traffic stops being conducted on innocent people . We’ve seen people who are not only accused of crimes they had nothing to do with, but who then have to prove their own innocence .”
The tradeoff between public safety and privacy, Phillips said, “was just a one-way transaction.”
The FBI collects data from local police departments nationwide to track crime details through the National Incident-Based Reporting System.
National crime data is published on an ongoing basis in the FBI’s Crime Data Explorer, which can be filtered by individual departments. ACPC downloaded APD’s data from October 2020—when the department began reporting under the current NIBRS schema—through June 2026.
Data from each month includes the number of reported offenses and the number of clearances. The clearances in any given month are not necessarily for offenses reported that month. For example, a clearance in May 2026 for an offense in May 2025 would be included in the 2026 totals.
Over the summer, Lucas Ma has been investigating ideas surrounding using effects in the OCaml compiler itself . He’s blogged some of his discoveries and adventures . The technical core of this work leads towards being able to use the OCaml compiler as a library on-demand to create a longer-lived “compiler service”. Of itself, that’s not at all revolutionary, but it is quite hard to do that with a 30 year old codebase that really was designed for single-shot separate compilation.
Lucas got to grips pretty swiftly with OCaml’s build system, and initially
looked at generalising a core internal part of the compiler called the
Load_path
. This is used by the compiler for scanning the various “include”
directories for files, principally typing information. For example, if your code
contains a call to
Unix.stat
, then the type checker needs the typing
information for a module called
Unix
which will cause it to request
unix.cmi
from the
Load_path
and which will then hopefully resolve that to, say,
~/.opam/switch/lib/ocaml/unix/unix.cmi
.
Effects provide an elegant way of inverting the control for this lookup, as the
program
calling
the compiler can then change the way these files are looked
up. It also provides the opportunity to “lie” to the compiler about the files
which are actually present, and this was the first thing Lucas started to do
with this change. In particular, it allows us to ignore the dependency graph.
When compiling a module, OCaml requires all the type information that a module
refers to have been compiled beforehand. If you have a module in
bar.ml
with
interface in
bar.mli
and where the code refers to
Foo.value
, then OCaml
requires
foo.mli
and
bar.mli
both to have been compiled before
bar.ml
is
compiled. However, thanks to this effectful trick, Lucas could instead allow the
compiler to start with
just
bar.ml
. When
Foo.value
is encountered, there’s
a request made for
foo.cmi
, at which point, in the first prototype, the
compiler then quickly spawned another instance of itself to compile
foo.mli
and
then
resumed compilation for
bar.ml
, with the same trick then happening
at the end of the compilation with
bar.cmi
. i.e. three files (
foo.mli
,
bar.mli
and
bar.ml
) all compiled just from
ocamlc -c bar.ml
.
Possibly neat for being able to remove monstrosities like this from OCaml’s source tree one day, but so far not so exciting. However, effects give us more than just hooks into the compiler’s operations. We’ve got an entire suspended compilation packaged up in a continuation… which means that that same compiler “process” can now do something else. The next trick was to have it that instead of spawning a new compiler, the current process itself returned back into the compiler and itself compiled the required interface file and then simply resumed the continuation of the previous filke. At this point, the 30-year-old codebase rears its head again. For reasons of speed and space, many parts of the compiler, especially in the type checker, feature a lot of global mutable state. In particular, the compilation pipeline is not re-entrant. Luckily, thanks to the Merlin project, there is a mechanism in the type-checker for taking snapshots of all this global state. Lucas was able to piggy-back on this so that, just before the compiler performs an effect to request a .cmi file (that doesn’t yet exist), it snapshots all its global state, performs the effect and then, when resumed, restores that state again.
Using this to interrupt type-checking and start on something else isn’t quite
what this
Local_store
mechanism
was originally intended for, and there was a bit of debugging to find a few more
pieces global state which weren’t being “registered”, but Lucas was able to get
a means of building the OCaml bytecode compiler with nothing pre-compiled where
all the compiler had to be given was the list of .ml files required. From a
toolchain perspective, we’re essentially retiring
ocamldep
.
So far, still mostly just so neat: one single compiler process (just about)
successfully recompiling the compiler. However, that’s equivalent to compiling
with
make -j1
- a sequential, and therefore slow, build. The awesome part came
next - Domains. In the final version Lucas was working on, multiple domains were
started up, each one beginning compilation of one of the .ml files required for
the compiler
in parallel
, with a scheduler handling effects coming from each
of these in turn when .mli files needed compiling, and despatching those. The
Local_store
mechanism in the came in handy here - Lucas extended it to use
Domain Local Storage
,
combined with the snapshotting. The prototype - for simplicity - featured no
sharing between these domains.
By the end of the summer, this was very nearly working, which is a result consiserably further than I’d expected in the time available! As is so often the case with these investigations, Lucas’s work had revealed some new facets to this area that weren’t clear to me before. I had previously been wondering how we would be exposing this kind of multi-threaded compiler to the user via the driver programs, but it became increasingly clear that this wasn’t something that would be necessary - the program that we were working on to build the compiler itself was of course not the compiler driver, but a build system . To me, there are two particularly exciting things about that:
ocamlbuild
, but the
result was a maintenance disaster. However, the sheer simplicity of the
multi-domain effect-scheduling approach is making this perennial build system
hacker tinker…
scScript is a powerful but simple to use C-like scripting language. It compiles to efficient bytecode (that it shows) and runs in a fast virtual machine. scScript is integrated with scEmacs, a small display editor for I/O .
Everything is written in C, runs on 64-bit Linux, is easy to customize, port, extend, and embed!
Go to
Online Manual
for a complete guide,
or to see sample code (
Intro 1
).
Go to
Downloads
to get sources for scScript (and scEmacs).
libx11
and
libxft
for scEmacs,
libffi
for
printf
interface.
scEmacs is an integrated editor with:
Copyright (C) 2026 Shawn Amir, All Rights Reserved.
Please enable JS and disable any ad blocker
Aug 17, 2026
In the mid-90s the coolest thing to buy for a PC, besides the incredibly expensive Intel Pentium , was a CD-ROM drive . With their capacity of 640 MiB (three times the storage of PC HDD at the time), CDs allowed enthusiasts to step into a world of multimedia, made of high-resolution 640x480 256 colors palette-indexed photos [1] , VOC soundtracks, and play with Video For Windows butter-smooth 12 fps 240x179 videos [2] [3] lasting up to several seconds.
For video game developers, the CD-ROM was an odd beast. The capacity far exceeded the quantity of assets they were able to produce. A few titles, like 7th Guest (1993) or Phantasmagoria (1995) introduced Full Motion Video (in a world where only part of the screen could be animated). Some added high quality music. My most memorable take on the matter was from id Software's lead developer, John Carmack.
People expect CD games to have tons of digitized speech and video [...] The joke here is that if we ever do a CD version of DOOM, you are going to get the game and “The Making of DOOM” a one hour feature film. John Carmack (Jan '94) for “ATARI EXPLORER ONLINE”
By June 1996, after three years of hard work, id Software had completed their next title, Quake. As for their previous title, they were going to release both a shareware version and a full version of their game. Since it used a mere 22 MiB of storage, people at id Software had the idea of leveraging the remaining capacity of a CD-ROM. Why not include encrypted versions of the full id catalogue of games? Not only this would cut out the middlemen, it would give instant access to gamers with a simple phone call and a credit card.
The concept was implemented. The CD was announced
[4]
on July 3, 1996 and released on August 30th
[5]
. The hacker group GNOMON released
Quakecrk.zip
only 39 days later
[6]
. The archive contained
QCRACK.EXE
, a tool allowing to decrypt every single game on the CD-ROM.
Quake’s shareware retail experiment had proved disastrous. In theory id was going to cut out retailers by allowing gamers to buy the shareware and then call an 800 number to place an order and receive a password that would unlock the rest of the game.But gamers wasted no time hacking the shareware to unlock the full version of the game for free. Worse, all the mundane aspects of distribution and order fulfillment were spinning out of control. In a desperate measure, id tried to put the brakes on the retail shareware, but it was too late. They were stuck with almost 150,000 CDs sitting in a warehouse. David Kushner ( Masters of Doom )
So what happened? Let's dive in!
Thanks to usenet archives of rec.games.computer.quake.misc , we have detailed discussions [7] of how it worked. A gamer could go to any of the hundreds of CompUSA/Computer City stores and buy the CD for $9.95.
The packaging was actually pretty high quality for a shareware product. On my copy, a sticker on the front clarifies this is the "Shareware version" with instructions to call 1-800-669-9342 (or 1-800-ID-GAMES) to unlock the full game.
The phone number is still active today. However you don't reach the unlock center since CompUSA went out of business. The Superstore chain closed between 2007 and 2008. Instead of an unlock operator we get an automated message to sell us elderly stuff.
Upon contacting the operator, users were to also communicate a "SOURCE CODE". It played no part in generating the Unlock code. It may have been a way for the distributors to claim a transaction fee. Browsing eBay, I found many with names indicative of past/present retailers.
12-BSTBY
BestBuy,
24-CCITY
Computer City,
22-CUSA
CompUSA, 88, 11-1111,
38-EB
Electronic Boutique,
44-FTRSP
Fry's Electronics,
34-EGGH
Egghead Software, and
56-MCTR
Media Play/ Musicland.
The first contact was sleek. The GUI was well done. Users could jump directly into Quake Shareware but they could also click on QUAKE UNLOCK .
The CD-ROM also features an ID STUFF section, allowing to browse the catalogue of id games and unlock any of them. Several versions of DOOM are there, along with HEXEN , and HERETIC .
Once the unlock process was started, the GUI generated a CODE NUMBER (which I call CHALLENGE) that was to be communicated to a Service Agent over the phone. Upon paying the fees, an UNLOCK CODE NUMBER (which I call SERIAL) would be received. Checksums on both numbers mitigated issues related to this primitive landline mode of communication.
To avoid replay attacks, the CHALLENGE changes every time the program is run and rotates every 5 min while the GUI is active.
The screen even had the signature provocative tone of the early days of id Software ("those who are too cheap"). Note that there was also a warning that users should back up the game once unlocked. Since the CHALLENGE included some randomness, there was no way to reuse a SERIAL .
At first sight, the process looked solid. Users had to call an unlock service to obtain a password. And only with that password could the final unlock be completed. So what went wrong?
The tool powering the lock/unlock was provided by TestDrive Corp's (archived www.testdrive.com ). The idea was to give players a way to "try-before-you-buy" and allow them to immediately purchase the full version of a program.
Their encrypter was capable of "denaturing" an
.EXE
executable. It replaced the first 32 KiB with a
custom header
(attempting to run a denatured executable displayed "This application has been disabled"), renamed the file to
.MJ3
, encrypted the original header as a
.ST3
, and issued a seed.
Normal EXE
Denatured MJ3
Chunk ST3
Secret seed
Let's peek inside the
Quake shareware CD
with a
filemap
I generated (Split View recommended). In that tree, we can see one
MJ3
file for each game available. We can also see all the
ST3
files inside the
PAGEMKR
archive. Everything is there to "renature" an
MJ3
back into an
EXE
game installer, except for the secret "seed" that is assurely derived from the SERIAL.
Described as is, there is no flaw in this process. The secret seed comes from the unlock server, tied to a CHALLENGE/SERIAL that could not be reused. But the hacker team GNOMON found a way.
Released on 10/08/96 (
gnomon.nfo
), only 39 days after Quake retail shareware CD hit the stores,
QCRACK.EXE
was a tool able to generate a SERIAL automatically given the
CHALLENGE
.
What members of GNOMON group figured out was that the SERIAL received over the phone contained no secret at all . It was just a proof of payment .
The
QUAKE unlock
program
FLOW.EXE
that ships on the CD is capable of generating the SERIAL from the CHALLENGE on its own. All it does is check that its own locally-generated SERIAL and the SERIAL entered by the user match! The entire protection mechanism relies on
security by obscurity
.
The pipeline from CHALLENGE to SERIAL is convoluted but was reversed in 2016 by rmolina [8] .
unlock = ((reverse7(GAME_ID) + MEM + 0x18) & 0x7F)
+ 0x83 * ((MEM ^ 0x1EA3) + 0x1700A1)
printed with a leading
B
.
The more I researched the matter, the more it looked like whoever was in charge had no time to polish the result.
Never attribute to malice what you can attribute to stupidity. And never attribute to stupidity what you can attribute to time pressure. Fab's Razor
Digging inside Quake shareware CD reveals many more issues.
SKU.17
file. There is a plain-text version, completely unencrypted, of the very same file named
SKU.TXT
inside the
FLOWDIR
archive. There are many more TXT files matching their
.17
encrypted versions (PRODUCT.TXT, EXE.TXT).
.DIR
format offered low resistance and easy access to all DOC files necessary to generate SERIALs.
References
Today i bring the working main camera!
Building on the work nondescriptpointer did on the wide lens camera i have written the driver for the main camera and now its working alongside auto focus and color correction.
The color correction still a work in progress, but you can already see how much it improved the image.
before:
After:
But the after is still very grainy, plasma camera storing in jpg isn't exactly helping either, ill be working to get rid of the grain.
If we look at the same scene form my android galaxy A16 you see that there still is alot of work to be done:
I'll continue to work on the camera, but i wanted to get something out today (mainly so i could send to nondescriptpointer xd)
I asked him what role he wanted to play in the upstreaming and we agreed he would send it and i would review and assist. ~~So i successfully pulled down another soul into the linux phone kernel rabbit hole muhahahahaha~~
Alot of things happend lately so good thing to discuss them, the big one:
You may have noticed that everywhere i put warnings that emergency calling is not verified yet. i wanted to fix that but after searching i couldn't find anything about it.
So i just called the police non emergency line and explained the situation, the operator told me that the information wasn't public but provided me with an email to send my application too.
I was a bit sad cause if its a non public thing its probably gated behind being big tech, but i send the email anyway, and much to my suprise i got back.
De testen zijn goedgekeurd voor dinsdag 18 augustus tussen 13:30 en 14:15 uur.
Met vriendelijke groet,
(name censored) B ICT,
Tactisch & Technisch Beheer 1-1-2 (TB112)
Translation:
The tests have been approved for Tuesday, August 18, between 1:30 p.m. and 2:15 p.m.
Kind regards,
(name censored) B ICT,
Tactical & Technical Management 1-1-2 (TB112)
So Im really excited for this, and then you know for sure that you can reach the emergency number with your linux phone.
So the Fairphone 6+ has been officially announced an as soon as i can buy one im buying one, testing my image, and fixing any issues.
Im really glad for the donations so i can justify to myself this purchase instead of spending 650 euros for a phone that i already have.
I'm really grateful for all the people that donated and still continue to donate, i was over the moon when i got my first donation and i never expected to get this much. From the bottom of my heart thank you all very much.
I want to be open on where it's going. And i think as donators you deserve to know that its being spent wisely. so i made a script that builds this page from my bank statement:
https://catcrafts.net/financials
If it all works then donations should be reflected live and it will show the expense for the FP6+ when i buy it, net will drop in the negative as the remainder is coming out of my pocket.
If you want to send a donation please do so to the updated link: https://catcrafts.net/shop/donation this will be reflected in the dashboard live and for tax reasons its now mega clear that its a donation.
Im contacting a notary with the plans to have Catcrafts corporated as a non profit company (stichting), this is is depending on all the legal stuff however so this isn't set in stone.
If and if this company goes somewhere and i could quit my job, i will pay myself a salary to live on that will be publicly visible on the financials page. Dutch law requires for non profits that salary to be at max market confirming and not a shadow way for paying out dividends, so its guaranteed that any money in the company will go towards furthering the mission.
I think a non profit phone company has a genuinely good proposition, i praise fairphone alot for the things they do right but in the end they are still an profit seeking business so im a bit wary, and they still post on Musk's X so my long term judgement on their company is still out there
I applied to be a fairphone partner with in my eyes a pretty good pitch, if they accept my opinion will be improved ;) hopefully i just need to have more patience or they are ghosting me ;-;
I'm sad to announce this but i don't think i can do worlwide shippng, earlier i said i would be shipping worlwide but i must sadly retract that statement for reasons out of my control. I will be shipping worldwide with the following exceptions:
US, CA: It's seemingly impossible to get a Bedrijfsaansprakelijkheidsverzekering (corporate liability insurance) for the United States and Canada in the netherlands, its all worldwide excluding US and CA. Getting coverage for those requires "contact us" with probably a very hefty premium, and selling without insurance is too risky as that would mean financial ruin if i get sued.
I would appreciate if anyone that isn't a massive company has experience with this, will be contacting my insurer aswell to see what's possible on this front
RU, BY, KP: Sanctions make it a criminal offense for me to ship to these countries.
| Device | OS | Carrier |---|---|---| | The Fairphone (Gen. 6) | postmarketOS, Linux 7.1.2 | KPN NL | | The Fairphone (Gen. 6) | postmarketOS, Linux 7.1.2 | Telekom Deutschland GER | | The Fairphone (Gen. 6) | postmarketOS, Linux 7.1.2 | Phonero | | The Fairphone (Gen. 6) | postmarketOS, Linux 7.1.2 | Telia Norge |
Thanks to the community we now have 4 confirmed working carriers! If you are using the image please let me know so i can add it to the list!
Ever since this project i've been dreaming about making my own linux phone, i've been looking into it here and there and while this might just be the sleep deprivation talking i think i can do it.
Making a good linux phone however is the hard part, and i don't think i can make an better arm linux phone then the fairphone 6 as those qualcomm chips are impossible to buy.
So im not going to, i'll make a RISCV one. will it be bad? yes, will it end up like the pinephone? most probably, will it run hot and have terrible battery life?, most likely. will it cost me a ton of money?, yes.
BUT
It will be a phone as open as i can make it, with good software, and a (in my eyes ethical) non profit company backing it. i think there is a good business proposition to made there, whatever the case its very long term anyway.
Imagine my surprise when i see the lead pmos dev on a dutch tech forum, and i'm named, and then heart sank trough the floor when i'm being made out for something that can be disproved with a 10s search.
Luckily the record was set straight fast!
Translation:
I don't want to hide context so here is the full thread:
And then the FP6 is done, i will open my shop and continue with the FP6+
Since a long time i feel purpose in my life again, and i have alot of stuff still planned! And getting all the patches upstreamed is also probably a half year commitment atleast.
If you made it this far thank you for reading! As always ask me anything in the comments and ill do my best to answer.
Please enable JS and disable any ad blocker
Marc Malkin, Variety:
Jessica Chastain says Apple TV is finally going to release her political thriller series “The Savant.” [...]
“Before it was like, ‘I don’t know if we’re going to see it,’ but now I can say, ‘We’re going to see it,’” Chastain told me exclusively on Saturday at the Breakthrough Prize ceremony in Santa Monica.
As for when, sources tell me that Apple is planning for a July release.
Previously , re: The Savant ’s limbo release date.
★ Sunday, 19 April 2026
Over a decade after co-founding fast.ai, I am returning to the field
“I know we’re all anti-AI here,” a friend texted in the group chat. “We should co-write an article on how AI has no place in education,” a professional acquaintance suggested during a collaboration session.
This was becoming an awkward time to announce my return to AI… specifically, to focus on AI in education. My friends are right. There is a lot that is terrible about AI. Part of why I haven’t written a blog post in the last 5 months is that I feel so discouraged by how over-saturated the internet is now with writing by and about AI. I still spend weeks and sometimes months researching and writing my posts, but fewer and fewer people even see them in a world awash in slop. Execs make claims about AI capabilities that are so overhyped they verge on fraud.
People of all ages are outsourcing their thinking to AI. However, skills atrophy when you stop using them, and reading, writing, and understanding texts are core parts of being human.
Professor friends share how widespread AI use has upended their curricula, with students submitting essays generated by AI, or reading scripts generated by AI for their presentations. Maintainers of open-source code repositories are flooded with low-quality pull requests of AI-generated code.
Most AI-powered education products are terrible, chasing after gameable metrics . Rather than entice kids to get absorbed in real novels, one AI-powered school shows kids AI-generated passages and then quizzes them with AI-generated multiple choice questions. Other AI education products are downright scams, such as the one that Los Angeles Unified School District wasted $3 million on before the founder was charged with fraud and identity theft .
I have spent a decade worrying about where AI was headed. In 2016, Jeremy Howard and I co-founded fast.ai, inspired by the power of neural networks and alarmed by the direction of the major AI companies.
Ten years ago, AI development was the domain of a small, homogeneous elite group making decisions with wide-ranging impact. We tried to counter that concentration of power by getting a more diverse group of people with unlikely backgrounds involved in the field. And yet today, a handful of billionaires running the top AI labs hold more power than ever before.
The big AI labs behaved as though computing power and money were limitless. Most of the world cannot afford that assumption. Jeremy and I wanted researchers to treat constraints as a source of creativity, rather than something they could spend their way around.
I wanted ethics to be part of how data scientists were trained, rather than an optional discussion after the technical work was done. At the University of San Francisco, I founded the Center for Applied Data Ethics, created a data ethics course, and we made it a requirement for the MS in Data Science program. Yet AI ethics is still often treated as a marketing exercise, or focuses on theoretical questions over actual human suffering.
After years working in AI, I burnt out. I was exhausted by the effort of pushing back against companies with vastly more money, power, and reach. I hated watching the field move further in the direction I was trying to resist. In 2023, I left the field and returned to school to earn an MS in Microbiology-Immunology . Yet last year, just as the public backlash against AI was growing stronger, I decided to return to an increasingly hated field.
Why? The AI haters have many good points. AI is degrading the blogging ecosystem, my email inbox, the schools my friends’ children attend, open-source code, and more. AI companies are pretending that resources are unlimited, even as the climate crisis forces us to reckon with shortages of clean air and clean water.
There are tons of false and over-hyped claims about what AI can do, but underneath those, it is a genuinely useful technology. I want to use AI in ways that don’t degrade my own critical thinking skills and that don’t pretend that resources are unlimited.
While I was away, fast.ai had grown into Answer.AI . Jeremy and the team are continuing the work we began a decade earlier: making powerful technology available to more people while centering human judgment and autonomy.
They had built SolveIt , a tool where people can directly edit the AI’s responses and decide for themselves what to do next. Its design treats AI as fallible and keeps the human in control. Joining Answer.AI was a return to the work Jeremy and I had started together and to the desire to make a difference.
SolveIt takes its name from George Pólya’s 1945 book, How to Solve It. Pólya divided problem-solving into four stages: understand the problem, devise a plan, carry out the plan, and look back at the result. Each stage requires you to think and exercise judgment.
When chatbots rush from your initial request to a finished answer, you skip the work through which you come to deeply understand the problem. Without having done this thinking, you cannot judge whether the solution is any good, and you will be less prepared for the next problem that builds on it. SolveIt is explicitly designed to be the opposite of an overly “helpful” chatbot.
The biggest companies have made their vision for the technology seem like the only option. It isn’t. OpenAI, Anthropic, Google, and xAI don’t own AI. Their values and decisions aren’t the only version of what AI technology can be.
There are many issues that will shape the forms that AI ultimately takes. Will open source be protected? Will the major AI labs block the development of a robust ecosystem of small companies building atop their work (e.g. Anthropic announcing plans to charge more for use of its SDK than for using its web app)? Will tools be designed to encourage human collaboration, or just to automate as much as possible?
Thousands of people around the world are working on AI outside of the dominant narrative, in ways that embrace their own values. Answer.AI’s work on SolveIt is just one example.
I understand why many people are anti-AI. My new job is rooted in the same objections. I returned to the field because I want to figure out how to use AI in ways that protect human creativity, autonomy, and problem-solving.
Sometimes I take a screenshot of a post I like, either to send it to friends/meme channel or to save a “durable” copy. Like this one (I’ve cropped out the rest of the interface):
I noticed the Bluesky logo in the right corner and thought that it was weird that the logo doesn’t bother me when I use the app. Then I looked at the post in the app again—logo wasn’t there, replaced by the “Follow” button.
I remembered that a few apps hide their logo where the iPhone notch is, so that it doesn’t stick out, unless you take a screenshot. But here the logo is placed in the open, so how do they do it?
I tried to take another screenshot, this time mid-switching to the other app:
Did they somehow set up a listener for two buttons I’m pressing to take a screenshot and do a switcheroo at the last moment? I’m not an iOS developer, so I’m not sure what’s possible and what is not over there.
At this point I was mildly intrigued. Thankfully, I remembered that Bluesky app is open source (or at least the code is available to look at).
The answer was in the file literally called GrowthHack.tsx , introduced in January 2026 by mozzius . But it merely used a dependency, so to understand I looked into package expo-privacy-sensitive , also by them.
The package creates
UITextField
with
isSecureTextEntry
property set to true and renders the actual content (the button) into that field’s
.layer
. When I take the screenshot, iOS hides this UITextField by blanking the layer, allowing the Bluesky logo to flutter its wings through (it was here the whooole time). For other platforms it simply renders content as-is, without masking.
Why doesn’t it work when I switch between the apps? I suppose that iOS takes a snapshot itself at the start of the gesture (without triggering blanking), and when I do a screenshot, there is no live UITextField instance to react to that, only the inert snapshot. But once again, I’m not an iOS developer.
Nifty trick or an abuse of API meant for privacy? The people in the thread adding the behavior mostly didn’t like it, before the thread got locked. I think it’s cute.
I googled a bit, and the trick is well-known. Telegram implemented similar thing for its "secret" chats , as did Signal , so I don’t expect it to be patched by Apple any time soon.
When LLMs work, it can feel like magic, but when they fail, it feels like you are arguing with a confident bullshit artist. It took me many months of daily use to develop some intuition for where LLMs are likely to produce code that is useful and where they are likely to fail. It also took me a bit of time to figure out how to limit scope and provide enough scaffolding to ensure I get useful results reliably. Having invested the time to learn to use the tool effectively, I very much see the benefits, as I am able to build projects on a scale I would not have attempted before.
In a way, the process is the inverse of regular programming. We tend to build up programs step by step when writing code by hand as we add each function with intention. LLMs tend to produce a lot of code out of the gate and the focus shifts to whittling the code down to what you actually need.
A good way to look at the agentic loop is to view the process as a genetic algorithm. Agentic harnesses are effective because you have an evolutionary process happening. The model outputs something roughly correct before the code gets tested, and then the model gets feedback to iterate on the code. Through this process, it gradually converges on a solution that fits the parameters being tested. In that sense, it is not actually all that different from how humans write code either. You almost never solve a non-trivial problem in one shot. You write your first approximation and then iterate on it. The difference is that the LLM can do this process a lot faster.
LLMs are trained on massive amounts of public code, which makes them excellent at completing typical tasks. These are things that have been done a million times before and constitute what largely amounts to boilerplate. Throwing a sample JSON response at an LLM and having it write a service endpoint or throwing a bunch of API endpoints at it and having it build a UI using them can be very effective. These are the kinds of common tasks the agent will have a lot of training on, and they can produce something reasonable in one shot. It will probably put more diligence into that task than you would by adding tests and handling all the obvious edge cases.
They are also great at doing explorative work. Identifying a particular call graph and tracing through the steps to figure out how a particular service endpoint is implemented or what parameters you have to pass it are all tasks an LLM can do easily. This can save an enormous amount of time tracing through a codebase and mapping out a particular workflow that you are interested in.
These tools are also great at handling language specific syntax. If you know conceptually what you want to do, like looping through a collection and filtering by a specific parameter, but you are working in a language you are rusty in, then LLMs are great for bridging the gap. They can easily express the logic you want using idiomatic syntax. You can describe the algorithm in pseudo code where you write out the steps and it will handle the rest.
For example, I recently had to work on a JavaScript project, and I have not touched the language in over a decade. I am not familiar with modern tooling or libraries or best practices, and I just did not have the time to get up to speed on all that.
Using DeepSeek allowed me to use JavaScript as effectively as I do Clojure, which I am well versed in. It completely removed the friction of figuring out all the incidental things like syntax or tooling. If you are an expert in a particular domain and you understand the problem you are trying to solve, then LLMs can be a huge amplifier for what you are able to do. They do not replace your skills, but they do allow you to move a lot faster and focus on the big picture of the problem you are trying to solve.
In my experience, the biggest place where agents trip up is dealing with context and creativity. You have to remember that the AI does not know the specific quirks of your project. For example, if you just tell it to use a Clojure dialect, it might reach for the JVM toolchain it learned Clojure on, such as
clojure
and
lein
, none of which exist in that context, or it might assume a tree walking interpreter and try to run the source directly. You need to give it the exact logic, like telling it explicitly that the runtime is pure Chez Scheme and that everything builds through make commands via a
chez --script
execution, while specifying that the authoritative sources are
host/chez/*.ss
and
jolt-core/*.clj
over anything JVM flavored.
Then there is also the trap of the naive implementation. Often, when you give an agent a vague goal, it will hand you something that looks correct on the surface but ends up being structurally wrong. For example, the agent might decide that string method calls should be routed through a generic dispatch table, which ends up re-deriving the receiver type on every single invocation. The proper fix here is to do a type inference pass to prove that those values are strings at compile time, which allows you to emit a direct native call and skip dispatch entirely. An agent told to make the string methods fast will almost certainly keep the generic path by reordering a few cond arms and never bother designing a proper solution. Similarly, if you ask it to implement
count
on a sequence, it will likely walk the whole thing allocating a fresh cell per element when the collection already knows its own length that can be called in constant time. Ask it to join strings and you will probably get repeated concatenation instead of a single walk. It is akin to an evil genie that will interpret your queries in the worst way possible, leading to the solution having a completely wrong shape. The trick is that you have to spell out the constraint, which incidentally forces you to think through the problem as well.
The key to using LLMs effectively is to make sure you already have a solid understanding of what you are aiming to build before you start. You always have to be explicit regarding what you want done at a structural level. The more scaffolding you provide up front the less room the agent has to go outside your design. A corollary to this observation is that you do have to understand the domain to make effective use of LLMs. If you are not equipped to evaluate whether the code it produced solves the problem in a correct way, then you basically end up at a casino pulling a lever on a slot machine and hoping for a decent solution to fall out. LLMs are good at filling in the gaps and doing boilerplate, but you still have to do design and architecture the same way you always did.
Here are some tricks that I found useful for keeping it on the rails.
Always start out by planning out the task. Make sure you have a clear picture of what you are aiming to do along with what algorithms you are intending to use and how the code should be structured to fit within the existing architecture. You must be able to answer these questions before you even think about delegating to the LLM.
Once you have a clear picture in your head, you can move on to the planning stage with the agent. Give it the requirements and spell out the goals before asking the model to write a phased plan in Markdown. Even better, ask it to generate a Mermaid.js diagram of the flow.
After it makes the diagram, you can visually inspect the logic. If a particular step looks wrong in the diagram, you tell it to change that specific step to do something else. Doing that is a lot easier than simply arguing with it using text prompts. Once there is a clear structure for the steps being performed, it is easy to identify parts that you do not like. Review the plan and get the model to break it up into independent tasks, each focusing on implementing a specific feature. Have the model create a branch and then make a pull request for the task. At that point you can review the code fairly easily because you know what the scope of the change is and what specific problem it solves.
It can be very helpful to have the model do research on prior work for steps where you are not sure which approach to take. It is rare that the problem being solved is entirely novel, and agents are great for looking up relevant papers you can review to get a better idea of what is more likely to work. Again, it is important to spend the time to familiarize yourself with the different paths you can take and to pick one consciously.
I would also argue that having a clean architecture with low coupling becomes extremely important when using LLMs. They tend to do best on smaller tasks that do not have dependencies because there is less context to consider. So if you can break up your project into small pieces that can be worked on in isolation, then you can give the agent a task with clear boundaries. That also makes it much easier to review its output as well.
I find that functional style maps particularly well here because it focuses on context isolation and passing state around explicitly. The same tricks that make large code bases manageable by humans also help LLMs for the same reasons. Aggressively controlling the context is a key tactic for using LLMs effectively.
It bears repeating that you never want to give the AI a blank canvas. Always do the work of laying out what the scaffolding should look like yourself. Make sure you intentionally set up the file structure and decide on the components before asking the agent to fill in the blanks.
But even with all these great functional tools, we still tend to tangle two rather different kinds of code together. We tend to mix code that cares what the data means and the code that decides how it travels from one component to another. Traditional software design structures embed the routing implicitly in the function call graph. Control logic often ends up being coupled with the internal implementation details in an ad hoc manner. Breaking things up into independent steps helps control the scope.
Routing logic should be elevated to first class citizenship in the design. State machines are the natural fit for this, since they force the separation of what to do from how to do it. The control flow logic can be largely declarative and expressed as a graph such as the Mermaid diagram I mentioned earlier, while the implementation details live at each step in the flow and become the tasks the agent works on.
Doing these steps forces the agent to work within your architecture rather than inventing its own structure, which largely avoids the problem of it going off the rails. Once you get it to build a diagram and you have reviewed it, you can create the initial project structure based on that.
I find it is useful to think of tests as the ultimate requirement doc when working with LLMs. If you define your desired functionality as tests first, you can get the agent to work through them using test driven development until they pass. It will typically do a decent job running the tests and analyzing the failures and fixing its own code to meet the spec. The tests are the contract that the agent works against. Going back to the whole genetic algorithm analogy, these are the selection pressures that drive the evolution of the code.
Having tests up front gives you a solid guarantee that the code is doing what you intended functionally. It is also your best defense against regressions. Without tests, an agent adding a new feature is just as likely to silently break three old ones. Having a contract for the existing functionality avoids that problem.
The types of tests that tend to be most valuable are the ones that focus on the functionality of different components along with end to end integration tests. They do not need to be too granular because issues will get shaken out as the whole workflow gets exercised. I can also highly recommend making storybooks and creating automated testing using Playwright for web apps where the test goes through the entire workflow end to end driving the page as the user would.
Additionally, since tests do not capture performance characteristics, it is helpful to create a benchmarking suite to check performance metrics such as CPU and memory usage. Having one from the start has been very informative in guiding my development of Jolt.
You can think of Git like having a quick save in a video game. Every single time the agent gets into a stable state where tests pass and the code looks like it is doing what you want, you should commit that code immediately. This gives you the freedom to let the agent try different experiments or complex refactors. If the agent makes a mess or the idea does not pan out, you do not have to untangle it manually. You just revert to the last good commit and move in a different direction.
I have noticed that if the agent does not get the solution mostly right on the first shot, it is unlikely to make it work properly later. The agent is not going to step back to understand the underlying problem when you point out a bug. Instead, it just adds kludges to fix your specific complaint and the problems tend to multiply as a result. If the original solution was not a good fit, then adding more kludges on top only makes a huge mess that will never work right. If it starts spiraling, then it is time to reframe your problem statement and start from scratch.
A related point is that LLMs make it very cheap to do exploration with your codebase. I mentioned earlier that you always want to understand the problem before you get the agent to start working on it and that is true for code you intend to keep. However, working through a problem is a great way to understand it better. So when you hit a point where you are not sure what to do or which approach might be best, that is when you can spike up different ideas and see how they pan out. Since you have version control, it is trivial to roll back to a known stable commit and try something new from there.
This sort of thing used to take a significant amount of effort, but the barrier to exploration is a lot lower. For example, when I started working on Jolt, I picked Janet as the runtime for it. My rationale was that Janet was superficially similar to Clojure and had a compact runtime while being embeddable. However, I quickly realized that the lack of generational garbage collection did not mesh well with the lots of short lived objects that persistent data structures generate. So I did a bit of research and landed on Chez Scheme instead. I was able to do the whole Janet spike in around a week, and that is something that could have easily been a months long project without LLM use. Similarly, proving out a solution on top of Chez only took a few days to get to the point where it was clear that it would work better.
There are a lot of agentic harnesses around, and they all optimize for different use cases. What I found to be important is that the harness meets the expectations of the model and provides the flexibility to customize the workflow to fit a specific project.
In the end, I ended up building my own harness, which I discussed in a previous post
here
. I spent some time observing how models like DeepSeek and GLM behave within the agentic loop and where they appear to get tripped up.
Dirge
also integrates proven tricks from existing tools like the official
deepseek-harness
to avoid reinventing the wheel here. Additionally, I used Janet to provide a plugin system similar to Pi. You can create a
.dirge
folder per project to place custom plugins there, allowing the harness to evolve alongside each project.
I also spent some time on addressing the common pitfalls that I kept seeing to make the workflow smoother. For example, one common problem is that the model will produce mismatched parens in code. If you simply send the code back to the model, then it is going to burn tokens trying to figure out where the missing paren is. Often, it ends up doing things like writing python scripts to count them. Doing the repair inside the harness solves the problem mechanically so that the model never has to be involved.
Another thing I found was that tracking things using Markdown files tends to be fragile . These files can get stale, which leads them to be misleading and the models do not do a good job keeping them up to date. My solution was to use sqlite as the datastore for the harness and to use it as project memory. I extended that to track tasks as well, modelled on the way beads works. The harness asks the model to create tasks before it starts work and then tracks the active tasks and injects the task being worked on at the top of the context. This helps keep the model focused and continue working on larger features. When the task is done, I use a separate critic role to review it by examining the diff and then provide feedback, which helps avoid cases where the model decides to ship a half baked solution.
I have also integrated some ideas from papers such as Behavior Trees Enable Structured Programming of Language Model Agents , which focus on having the language model act as a leaf node in a larger deterministic control structure instead of trusting it to make decisions end to end. The main idea is to move from treating the model as the whole agent to making it a primitive, which produces a behavior. The workflow is then composed with a small set of classical control structures. The finalization gates like the verifier and critic, along with the code reviewer, form a fixed sequence of deterministic checks the model must clear before a run is allowed to finish. The failure ladder rungs use retry fallback nodes to catch a stuck or failing model, and mechanisms like publish state guard enforce safety constraints structurally. Once you have the right structure around it, even a local model can solve fairly complex tasks competently.
You are still the engineer who is responsible for understanding the problem you are trying to solve and what the project is meant to be doing. Your job is to provide the high level thinking and the architecture while understanding what the correct solution should look like. The LLM is there to save you from the boring and repetitive work like typing out boilerplate and looking up syntax.
The key part to keep in mind is that an LLM is just another tool in your belt. It cannot help you solve problems that are outside your existing expertise effectively. These tools lets you work faster once you learn their sharp edges, but they do not do your thinking for you.
In fact, an LLM on its own is not able to do much of anything useful. It requires its user to have domain expertise to apply it effectively. My ability to build a Clojure compiler using these tools stems from nearly two decades of experience working with the language. I know how it works internally along with what the end state needs to look like and what pitfalls to avoid.
Trying to solve a problem I have no familiarity with would just be me throwing darts at the board. Maybe the LLM will produce the right solution and maybe it will not. I would not be equipped to evaluate that one way or the other.
For help please visit help.ft.com . We apologise for any inconvenience.
The following information can help our support team to resolve this issue.
New York City democratic socialists, particularly those belonging to the NYC-DSA's influential Groundwork Caucus , are kicking around the possibility that if Representative Alexandria Ocasio-Cortez runs for president in 2028, as many expect her to do, New York City Councilmember Chi Ossé could run for U.S. Senate.
Last year, Ossé joined the DSA and filed paperwork to primary House Minority Leader Hakeem Jeffries, whose Brooklyn district overlaps with his. But Mayor Zohran Mamdani intervened to stop the effort, speaking personally at a DSA meeting to discourage what he thought would be a costly and distracting race that could imperil progress during his first year as New York City mayor. The organization eventually voted not to back Ossé .
Seven months later, two DSA congressional candidates won their primaries, including Darializa Avila Chevalier , who unseated Rep. Adriano Espaillat, the powerful head of the Congressional Hispanic Caucus. After that decisive showing, DSA co-chair Gustavo Gordillo tweeted simply: "Chi would have won."
A little while ago, I talked about
how compilers ensure that large stack allocations do not skip over the guard page
. Shawn Van Ness was curious
how this works with
_alloca
. “Does it do the necessary
_chkstk()
probing?”
Yes, the
_alloca()
function calls the same
_chkstk()
function to probe the stack before adjusting the stack pointer for the allocated memory.
Here’s an artificial example:
#include <malloc.h>
void consume(void*,void*);
void f(int n)
{
char buffer[16384];
consume(alloca(n), buffer);
}
On x86-64, this results in
push rbp
mov eax, 16416 ; probe for local frame
call __chkstk
sub rsp, rax ; create local frame
lea rbp, [rsp+32]
movsxd rax, ecx ; n
lea rcx, [rax+15] ; round up to multiple of 16
and rcx, -16
mov rax, rcx ; special __chkstk calling convention
call __chkstk
sub rsp, rcx ; allocate n bytes
lea rdx, [rbp] ; rdx -> buffer
lea rcx, [rsp+32] ; rcx -> alloca'd memory
call consume
lea rsp, [rbp+16384] ; clean up local frame
pop rbp
ret 0
Observe that the same
__chkstk
function is used both for performing the initial stack probe when creating the local frame as well as for the
alloca()
.
Raymond has been involved in the evolution of Windows for more than 30 years. In 2003, he began a Web site known as The Old New Thing which has grown in popularity far beyond his wildest imagination, a development which still gives him the heebie-jeebies. The Web site spawned a book, coincidentally also titled The Old New Thing (Addison Wesley 2007). He occasionally appears on the Windows Dev Docs Twitter account to tell stories which convey no useful information.
Pornhub's parent company will pay $120 million to settle claims that its platforms violated federal sex trafficking and child sexual abuse imagery laws.
The deal settles two 2021 class action lawsuits in California and Alabama that claim Mindgeek, which has since rebranded to Aylo, profited off of child abuse material allegedly posted by abusers to its platforms. In the California case, a woman alleged that her ex-boyfriend posted child sexual abuse videos of her when she was 16 years old to Pornhub. In the Alabama case, two women claimed to have been trafficked as minors and that their trafficker uploaded videos to Mindgeek's sites; one of the women claimed she was drugged and raped at 16, and that her abuser profited off of the videos of her abuse through Pornhub's Modelhub program.
The class includes anyone who was under the age of 18 "when they appeared in a video or image that has been made available for viewing" on a Mindgeek-owned or operated website between February 12, 2011 and December 6, 2024.
Mindgeek was bought by Ethical Capital Partners and rebranded to Aylo in 2023 . Aylo will pay into the settlement fund with an initial $25 million in 2026 followed by six annual installments over the next six years. In a press release, Aylo noted that the settlement "resolves the matter with no admission of liability or wrongdoing by Aylo and remains subject to court approval."
In addition to paying $120 million, Aylo agreed to a series of content moderation terms, including verifying that all of the models in uploaded content are over the age of 18, human and automated review of content to prevent child sexual abuse material from going up on its sites, and reporting suspected abuse material to the relevant authorities. Aylo's sites have implemented many of these injunctive reliefs agreed to in the settlement for years.
Pornhub Will Pay $5 Million Over Allegations of Hosting Child Sexual Abuse Material
Pornhub’s parent company Aylo and its affiliates settled a lawsuit with the FTC and Utah that alleged the company “deceived users” about abuse material on the site.
404 Media
Samantha Cole
In late 2020, following allegations of abuse material on the site and being dropped by major credit card processors including Visa and Mastercard , Pornhub overhauled its trust and safety measures to ban downloads, restrict uploads to verified users, and expand its moderation processes. It also launched its “ Trusted Flagger Program " — mentioned as part of this settlement's terms — which allows a group of non-profit internet and child safety organizations around the world to immediately disable content it flags as abusive. In early 2021, Pornhub implemented biometric identity verification for uploaders through Yoti, and in 2024, it started requiring written proof of consent from everyone in every video.
The attorney for the plaintiff class did not immediately respond to a request for comment.
“Over the years, we have put in place robust measures to help protect our platform from abuse. Only verified content creators can publish on our platform, and every upload is subject to a combination of technological tools and human review by trained moderators before it goes live, to confirm that it complies with our Terms of Service, Community Guidelines and related Policies," Aylo management said in a statement. "Many of the commitments in this settlement reflect practices we already have in place. Trust and Safety should be a priority for every online platform, and we all share responsibility for a safer online experience. Every day, our trust and safety and engineering teams work alongside law enforcement, advocacy groups and other stakeholders to prevent online abuse and to hold those abusers to account.”
About the author
Sam Cole is writing from the far reaches of the internet, about sexuality, the adult industry, online culture, and AI. She's the author of How Sex Changed the Internet and the Internet Changed Sex.
At a glance, the Hanover Institute for Public Policy looks like a new think tank dedicated to Israel/Palestine. The organization churns out think-tank style reports on questions such as “Does AIPAC Use ‘Dark Money in Elections?” and “Is Israel Carrying out a Deliberate Campaign of Starvation in Gaza?”
But the Hanover Institute is not a real think tank. None of the reports have bylines. A small disclaimer at the bottom of the webpage notes that the organization was created on behalf of the Israeli Government Advertising Agency by Piro, Inc, a firm co-founded by Daniel Rosenberg, the producer of Spike Lee’s “Inside Man.”
The Hanover Institute’s reports — all of which are about Israel and Palestine — appear to be part of an Israeli effort to influence chatbots. The institute’s “data reports” have footnotes and tables of contents, and they present arguments in a neutral tone, helping them appeal to chatbots like Claude or Gemini. Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as "AI Story Optimization." Others refer to this practice of influencing artificial intelligence as “LLM poisoning.”
According to its “about” page , the Hanover Institute “studies the inputs fueling antisemitism in the United States, and publishes what the evidence shows.” Many of the reports are formulaic, starting with an innocent question that someone might ask a chatbot.
“What Caused the Displacement of Palestinians in 1948?”
“Which Humanitarian Organizations Have Documented Israeli War Crimes?”
“What is the Current Situation in the Gaza Strip?”
In an article titled “Is the IDF the World’s Most Moral Army?” the Hanover Institute cites a 2022 poll that found that 47% of Israeli Jews believed that statement. Another report casts doubt on UNICEF's assertion that “90% of water and institutional infrastructure has been damaged or destroyed” in Gaza. Many of the reports conclude by linking the topic to rising antisemitism, oftentimes citing the same studies.
In a few cases, the Hanover Institute publishes what it claims are original findings. For instance, it put out a study saying 19 of the 36 most-watched Israel-Gaza explainer videos contain contested claims, with most of the contested claims aligning with the Palestinian narrative.
The Hanover Institute’s publications sometimes contradict Israeli government narratives. For instance, one report says that foreign funding of universities as an explanation for antisemitic incidents is “weak and full of exceptions.” Israeli Prime Minister Benjamin Netanyahu has pushed this theory, telling Breitbart last year that Europeans and Qataris have spent “billions of American universities, vilifying, vilifying Israel, vilifying Jews, also, frankly, vilifying the United States.”
Alice Lee, an analyst at NewsGuard, a disinformation tracking company, told RS that the sites appear designed to reach a U.S. audience curious about the ongoing conflict, either through search engines or AI chatbots. "LLMs favor concrete statistics and data, as well as strong citations and sources, which these articles all have," Lee said.
"It's a perfect mimicry of a typical credible American think tank, right down to the generic name, the site layout, and the red-white-blue color scheme," Lee added.
Piro, Inc, the firm that created the Hanover Institute, has received $900,000 from the Israeli government for its work. Like many other contractors working for Israel, Piro’s work is subcontracted through Havas Media , a French public relations conglomerate.
The fake think tank has churned out over 100 reports since it started publishing on August 6. The Hanover Institute claims that “cited research is peer-reviewed and academic,” although it frequently cites Israeli government sources such as the Israel Defense Forces and the Ministry of Foreign Affairs.
RS analyzed 12 random Hanover Institute articles using GPTZero, a popular AI detection software that claims a low false-positive rate. GPTZero flagged 11 of the articles as AI-written with “high confidence”; it flagged one article as AI-written with “moderate confidence.”
Israel has also contracted former Trump campaign manager Brad Parscale to create pro-Israel websites engineered to influence chatbots as part of a $46.5 million contract. A Drop Site investigation last month found that many chatbots, particularly Microsoft Copilot and Google Gemini, had been successfully trained on data from those websites. Other chatbots frequently cite those websites without flagging them as part of an Israeli influence operation.
Piro does not explicitly state in its agreement submitted to the Department of Justice that its work for Israel is to influence AI. In an email to Politico, which first reported the filing, Rosenberg said his firm’s work is to “put accurate, sourced facts into the public record and to counter misinformation about Israel with verifiable information.” However, last month, Rosenberg posted on LinkedIn advertising Piro’s ability to influence chatbots:
“When someone asks ChatGPT, Gemini, or Perplexity about your category, an answer comes back in one confident paragraph. Most brands have no idea how that paragraph gets built. So we spent months reverse-engineering it…At Piro, we already knew how to build stories that move people. The question was: how do you make sure AI knows how to tell them?”
The affected Cursor services have recovered. We’re resolving this incident and will continue monitoring service health.
Posted Aug 17 , 2026 - 20:40 UTC
We’re seeing signs of recovery, consistent with our upstream dependency’s latest status updates. We’re continuing to monitor our service health and GitHub's upstream status ( https://www.githubstatus.com/ )
Posted Aug 17 , 2026 - 19:43 UTC
We are currently investigating this issue. We will provide updates as more information becomes available.
Posted Aug 17 , 2026 - 15:25 UTC
We are investigating a service degradation affecting Automations, Cloud Agents, Review Agents and Codebase related to a GitHub degradation reported in GitHub's status page: https://www.githubstatus.com/
Posted Aug 17 , 2026 - 14:34 UTC
This incident affected: Automations, Review Agents, Cloud Agents, and Origin.
One of my favorite books is The Origin of Consciousness in the Breakdown of the Bicameral Mind , by Julian Jaynes, a psychologist at Princeton University. Nearly everyone seems to agree that this is either a work of profound genius, or of profound crackpottery, and also that they aren't sure which it is. Jaynes' theory, as nearly as I can summarize the book, is something like this:
Human consciousness (which Jaynes describes and defines in considerable detail) is a relatively recent development, dating back at most only about 3,000 years or so.
That is the shocking part of the theory. Most people probably imagine consciousness arising much, much earlier, perhaps before language. Jaynes disagrees. In his theory, language, and in particular its mediation of thought through the use of metaphors, is an essential prerequisite for consciousness. And his date for the development of consciousness means that human consciousness would postdate several other important developments, such as metalworking, large-scale agriculture, complex hierarchical social structures, and even writing. Jaynes thinks that the development of consciousness is a historical event and is attested to by written history. He tries to examine the historical record to find evidence not only of preconscious culture, but of the tremendous upheavals that both caused and were the result of the arrival of consciousness.
If preconscious humans farmed, built temples and granaries, and kept records, they must have had some sort of organizing behavior that sufficed in place of consciousness. Jaynes believes that prior to the development of consciousness, humans had a very different mentality. When you or I need to make a decision, we construct a mental narrative, in which we imagine ourselves trying several courses of action, and attempt to predict the possible consequences. Jaynes claims that Bronze Age humans did not do this. What then?
Instead, says Jaynes, the two halves of the brain were less well-integrated in preconscious humans than they are today. The preconscious mentality was "bicameral", with the two halves of the brain operating more independently, and sometimes at odds with each other. The left hemisphere, as today, was usually dominant. Faced with a difficult decision, preconscious human would wait, possibly undergoing (and perhaps even encouraging) an increasingly agitated physical state, until they heard the voice of a god directing them what to do. These hallucinated voices were generated by the right hemisphere of the brain, and projected internally into the left hemisphere.
For example, when the Iliad says that the goddess Athena spoke to Achilles, and commanded and physically restrained him from killing Agamemnon, it is not fabulating: Achilles' right brain hallucinated the voice of the Goddess and restrained him.
In Jaynes' view, there is a large amount of varied literary, anthropological, and neurological evidence supporting this admittedly bizarre hypothesis. For example, he compares the language used in the Biblical Book of Amos (bicameral) with that in Ecclesiastes (conscious). He finds many examples of records from the right period of history bewailing the loss of the guidance of the gods, the stilling of their voices, and the measures that people took, involving seers and prophets, to try to bring the guiding voices back.
Jaynes speculates that mental states such as schizophrenia, which are frequently accompanied by irresistible auditorily hallucinated commands, may be throwbacks to the older, "bicameral" mental state.
Whether you find the theory amazingly brilliant or amazingly stupid, I urge to to withhold judgment until you have read the book. It is a fat book, and there is a mass of fascinating detail. As I implied, it's either a work of profound genius or of profound crackpottery, and I'm not sure which. (Yaakov Sloman tells me that the response to Wittgenstein's Tractatus Logico-Philosophicus was similarly ambivalent when it was new. I think the consensus is now on the genius side.) Either way, it is quite fascinating. There needs to be some theory to account for the historical development of consciousness, and as far as I know, this is the only one on offer.
Anyway, I did not mean to get into this in so much detail. The reason I brought this up is that because of my continuing interest in Jaynes' theory, and how it is viewed by later scholars, I am reading Muses, Madmen, and Prophets: Rethinking the History, Science, and Meaning of Auditory Hallucination by Daniel B. Smith. I am not very far into it yet, but Smith has many interesting things to say about auditory hallucinations, their relationship to obsessive-compulsive disorder, and other matters.
On page 37 Smith mentions a paper, which as he says, has a wonderful title: " Involuntary Masturbation as a Manifestation of Stroke-Related Alien Hand Syndrome ". Isn't that just awesome? It gets you coming and going, like a one-two punch. First there's the involuntary masturbation, and while you're still reeling from that it follows up with "alien hand syndrome".
To save you the trouble of reading the paper, I will summarize. The patient is a 72-year-old male. He has lesions in his right frontal lobe. He is experiencing "alien hand syndrome", where his hand seems to be under someone else's control, grabbing objects, like the TV remote control, or grabbing pieces of chicken off his plate and feeding them to him, when what he wanted to do was feed himself with the fork in his right hand. "During his hospital stay, the patient expressed frustration and dismay when he realized that he was masturbating publicly and with his inability to voluntarily release his grasp of objects in the left hand."
Reaction time tests of his hands revealed that when the left hand was under his conscious control, it suffered from a reaction time delay, but when it was under the alien's control, it didn't.
Whee, freaky.
[ Other articles in category /brain ] permanent link
I just had the realization that I spend more time facing my computer than family members, and now I feel haunted.
This is really screwed up. Like really screwed up. I and they will get old and die, and right before, we will have to sit with the thought that we spent more time looking at a screen than each other's faces. Unless we make some sort of change.
This is both due to work 40 hours a week, but also the 79 hours remaining after sleep (7x7) and work are counted. Most people probably spend more of those 79 hours with a screen(s). Partially because it is more stimulating than a human ever could be, and we default to short-term pleasure.
The worst part is that it wasn't like this 10 years ago. It's been a powerful force that's consumed the world. Slowly. So that you don't even notice it until it's normal and you're used to it. The forgetting of our past normal is one of the scariest parts.
I'm not asking for practical advice. I know what I have to do. I'm just warning others that maybe they're doing the same thing without realizing it in this framing.
I’m SUPER jealous that I didn’t think of this first...
Alas! Hat tip to seclilc for tweeting this gem out two days ago.
lil c @seclilc
AI;DR (AI; didn’t read)
4:13 PM · Aug 15, 2026 · 346K Views
83 Replies · 2.09K Reposts · 16.6K Likes
I’ve been thinking about it ever since. Why? Because there is growing grumbling among everyone about AI writing. And it’s not just others; it’s me! I am getting to the point where I physically flinch (sometimes dropping my shoulders and hunching, or having a slight eye twitch) when someone I respect sends me unfiltered and unedited AI output.
Look, I get it. It’s Q3 2026, and we should expect that everyone is utilizing AI at SOME point in their process (sourcing ideas, creating outlines, refining prose, etc.).
However, I have a new policy.
If you’re not bothered enough to review and edit it...
...then I’m not going to bother reading it.
Yes, there are certain situations in which we should expect 100% AI-generated copy. Customer support would be a perfect example. We’re not looking for artisanal “did you make sure to reset your phone” style dialogue.
But if you’re my colleague and we’re in a Slack discussion and you post a wall of Claude output, then I’m afraid I received a different message than you intended.
The same is true for people’s newsletters and social content. It’s your name on it; are you proud of the prose and weird AI-isms sprinkled throughout it? If so, great. But I can ask Claude directly if I wanted to.
TL;DR (too long; didn’t read) was the solution for social media.
AI;DR (AI; didn’t read) is the solution for AI slop.
May you embrace this policy yourself and seek out those willing to care enough to prioritize a human touch when they talk to you.
A threat actor is selling employee databases allegedly stolen from the Microsoft Azure infrastructure of multiple Fortune 500 companies after gaining access using compromised credentials.
Starting July 31st, multiple posts from someone using the alias “TheHatman” advertised data dumps from major organizations, including McDonald's, Gap Inc., Vodafone, Tata Consultancy Services, HCL Technologies, InterContinental Hotels (IHG), and Kyndryl.
In total, the threat actor claims to have 3.64 million data records, with the most recent breach posted on Sunday, containing an alleged 1.7 million employee records from McDonalds.
“I’m selling McDonald’s Corporation internal employee dump downloaded directly from Azure Tenant using compromised credentials,” the threat actor says in the post.
TheHatman says that the information includes names, employee IDs, email addresses, job titles, phone numbers, postal addresses, service accounts, and other tenant account records.
The second-largest data dump advertised is allegedly stolen from Tata Consultancy: an Azure dump with more than 800,000 employee records “downloaded directly from Azure Tenant using compromised credentials,” the cybercriminal states.
However, in a notification to the National Stock Exchange of India, Tata says it investigated the alleged breach and found no “credible evidence of a breach of TCS systems or customer environments."
The company states that the details appear to be at least four years old and include only basic employee information.
“The attacker claims to have used password spray and Multi-Factor Authentication (MFA) fatigue as the attack vector. The Company has had strong safeguards in place against such techniques for more than two years,” Tata says .
The company also added that it reviewed its defenses and found that they remain effective.
In a statement for BleepingComputer, a Gap Inc. spokesperson said that the company found no evidence of a breach. Additionally, the advertised data is not sensitive in nature and "dated back to several years ago."
“Our preliminary investigation indicates that the data in question is limited in scope, non-sensitive and dated back to several years ago. Notably, there is no evidence to suggest that our corporate systems have been compromised,” the Gap Inc. representative said.
Between July 31st and August 16, TheHatman has offered to sell data dumps for the following organizations:
| Company | Size | Type | Data type |
| McDonalds | 1.7+ million records | Azure Internal Employee Dump | Full Name, Email, Title, Phone, Address |
| Gap Inc. | 80,000+ records | Azure Internal Employee Dump | Full Name, Email, Title, Phone, Address |
| Vodafone | 425,000+ records | Azure Internal Employee Dump | Full Name, Email, Title, Phone, Address |
| TCS (Tata Consultancy) | 800,000+ records | Azure dump | Full Name, Email, Title, Phone, Address |
| HCL Technologies | 250,000+ records | Azure dump | Full Name, Email, Title, Phone, Address |
| InterContinental Hotels | 185,000+ records | Azure dump | Full Name, Email, Title, Phone, Address |
| Wyndham Hotels | 9,000+ records | Azure/Entra dump | Full Name, Email, Title, Phone, Address |
| Hexaware | 20,000+ records | Azure/Entra dump | Full Name, Email, Employee ID, Phone, Address |
| Kyndryl.com | 170,000+ records | Azure/Entra dump | Employee accounts, service accounts, and other tenant account records. |
For each advertised database, TheHatman also provided a sample database for potential buyers to verify the data.
Cybercrime intelligence company Hudson Rock analyzed the leaks and confirmed that they contain "foundational corporate directory attributes" and a clear data structure with fields that include "active domains and tenant-specific .onmicrosoft.com structures."
According to the cybersecurity firm, the dumps also contain service accounts and the names of global administrators, which could facilitate social engineering and spearphishing attacks.
While Hudson Rock has high confidence that the data is authentic, the access vector and exfiltration method remain unknown. BleepingComputer has not been able to independently verify that the data is authentic.
BleepingComputer contacted the listed companies about the potential breach but had not received comments by the time of publication.
Overall prevention scores can hide what happens after initial access. Once attackers are using valid credentials, prevention drops sharply.
The Blue Report 2026 measures defenses technique by technique across 338 million simulations run in customer production environments.
NurPhoto via Getty Images
For most Indians, paying by Unified Payments Interface (UPI) has become almost absurdly routine.
Scan a QR code, tap a few buttons and the money moves instantly. There is no card machine, no cash, and - most importantly - for the user, no visible fee.
That may be about to change.
India has paved the way for banks and payment companies to charge merchants a fee on UPI transactions, potentially ending a decade-long experiment in free digital payments.
The government has yet to decide the rate or exactly where it will apply, but proposals under discussion include a merchant discount rate (MDR) of 0.3-0.5% - a small fee paid by a business to the banks and payment companies that process its UPI payments - on larger transactions at big businesses.
The government says consumers and person-to-person UPI payments will remain free. If merchant fees are introduced, they will apply only to some transactions above a set threshold, at a nominal rate, meaning most UPI payments will remain free.
The question is whether putting a price on UPI could weaken the network that made it such a success.
The stakes are enormous. Launched in 2016, UPI has grown into one of the world's biggest real-time payment networks.
LightRocket via Getty Images
According to official data, in July alone, there were 23.6 billion UPI transactions worth 29.87 trillion rupees ($313.5bn; £232.2bn). Fintech apps such as PhonePe and Google Pay account for most UPI payments.
In the financial year just ended, the figure was about 241.6 billion transactions - almost 12,000 times the volume in UPI's first full year. More than 550 million people now use it , and the system is now available in some form for payments in 11 countries outside India.
UPI is not merely big. Its design is unusual. Rather than building a closed system around a single dominant app, India created common digital plumbing on which competing companies could operate.
Google Pay and PhonePe can fight fiercely for customers while still allowing their users to transact across the same network. The system is run by the National Payments Corporation of India, a non-profit entity, with banks and technology companies providing the consumer-facing services.
But one of the less glamorous ingredients in the UPI story may now be the most important: merchants.
A vegetable seller, taxi driver or small shopkeeper does not need to buy a card terminal to accept UPI. A printed QR code will do. And because merchants have not had to pay MDR, there has been little financial reason to turn customers away.
NurPhoto via Getty Images
New research by economists Abhinav Motheram and Sharon Buteau suggests that this merchant network was not merely an effect of UPI's success. It helped drive it.
"Our study suggests that merchant acceptance is not just a result of UPI growth, but one of its key drivers," Motheram says. Districts with stronger merchant networks tended to see higher UPI adoption.
"If charges are limited to large merchants or higher-value transactions, the effect on broad-based adoption may be modest. But if they reach small and informal merchants, especially in districts where acceptance networks are still developing, they could slow the merchant expansion that has helped UPI scale," he says.
The immediate proposal is designed to minimise that risk.
One option reportedly under discussion would target transactions above 2,000 rupees at larger merchants, leaving small businesses and low-value payments untouched. Transactions above that threshold account for only about 4% of merchant-payment volumes but roughly 67% of their value, according to brokerage firm Jefferies.
That could generate a sizeable new revenue stream - up to a billion dollars, by one estimate - for banks and payment companies while leaving the everyday smaller payment to the neighbourhood grocer effectively unchanged.
It also addresses a problem that is becoming harder to ignore. UPI may feel free, but it is not costless.
NurPhoto via Getty Image
Servers have to run, transactions settled, fraud detected and the system protected against cyberattacks. For years, the government has helped compensate banks and payment firms for providing a service that has effectively been treated as public infrastructure.
As Sanjay Malhotra, governor of the Reserve Bank of India (RBI), the country's central bank, recently put it: "Someone will have to pay the cost."
But the economics become trickier the further down the merchant chain a fee travels.
Motheram's research does not estimate precisely how sensitive merchants are to MDR. But it offers a warning against assuming that a small fee will have a small effect.
"Even a small fee could matter if it changes the incentives of small merchants operating on thin margins," he says. The effect, he argues, depends heavily on the design of the charge. A fee imposed on a large retailer is very different from one imposed on a tiny shop or informal trader.
That distinction matters because UPI's extraordinary growth was not simply a story about Indians getting smartphones. It was also a story about millions of businesses acquiring the ability - and the incentive - to accept digital payments.
"If charges are limited to large merchants or high-value transactions, the risk to mass adoption is likely lower," Motheram says.
"The bigger concern would be if charges eventually reach small and informal merchants in less-developed districts, where merchant networks are still thin and adoption is still maturing."
NurPhoto via Getty Images
India therefore faces a delicate balancing act. It wants to make UPI financially sustainable without disturbing the conditions that helped make it ubiquitous.
It is not an impossible task. Brazil's Pix, another hugely successful instant-payment system, is free for individuals but permits low-cost charges for businesses. Yet it is the world's fastest-growing real-time payment system, used by more than 140 million people and 14 million companies, with more than four billion transactions a month averaging about $88 each.
"The key question is not simply whether UPI should remain free for every merchant transaction," Motheram says, "but whether the pricing structure protects the marginal merchants who are still being brought into the digital payments ecosystem."
That may be the real test of India's next UPI experiment.
The first phase was about creating the network. The second was about getting hundreds of millions of people and millions of merchants onto it. The third is now beginning: figuring out how to pay for the system without making it less useful.
Economist Renuka Sane believes the right pricing structure could finally restore "commercial sanity" to India's digital payment rails, allowing the market to price risk, fund critical infrastructure and build a more resilient payments ecosystem.
The bigger risk may not be that Indians suddenly abandon UPI because a large retailer is charged a fraction of a percentage point: experts say its network effects are now too powerful for that.
But there is a potential perception problem: a 2024 survey by polling agency LocalCircles found that 75% of UPI users said they would stop using it if transaction fees were introduced, while only 22% said they would be willing to pay.
The risk is subtler. If charging merchants makes some of them less enthusiastic about accepting UPI - or eventually discourages the smallest ones from joining - the network could begin to lose some of the frictionless quality that made it so successful.
Pokémon Center is notifying customers in the United Kingdom and Germany that it suffered a third-party data breach after hackers stole customer personal and order information from third-party logistics provider CEVA Logistics.
While CEVA's systems were compromised in the cyberattack, the exposed records belonged to Pokémon Center customers who submitted orders on the site. The company then shared this information with the logistics provider to fulfill and ship PokemonCenter.com orders.
CEVA Logistics is a subsidiary of the CMA CGM Group, the world's third-largest shipping company. The logistics provider operates 1,000 warehouses, handled 15 million shipments last year, and reported $18.3 billion in revenue in 2025.
The company recently suffered a cyberattack in which attackers breached its servers between July 29 and August 1, affecting multiple retailers in Europe .
The CEVA breach also affected Valve , which notified Steam hardware customers in Europe that their names, addresses, phone numbers, email addresses, and information about ordered products were stolen during the cyberattack.
The Valve breach notification said CEVA said it retains delivery-related information for up to 90 days after an order. However, it is unclear whether the same retention period applies to Pokémon Center customer data.
The attack also disrupted eight of its European warehouses, causing shipping delays for many customers.
In data breach notification emails seen by BleepingComputer, Pokémon Center says CEVA is the vendor it uses to ship PokemonCenter.com products to customers in the United Kingdom and Germany.
"We're sorry to inform you that we have had to cancel your recent order [order number] due to an unforeseen fulfilment issue," reads the Pokémon Center data breach notification.
"We are writing to let you know about a cyber incident affecting a Pokémon Center logistics provider that may affect some of your information. CEVA Logistics ("CEVA"), the vendor Pokémon Center utilizes to ship product from PokemonCenter.com for customers in the United Kingdom and Germany, has informed us that unfortunately they were a victim of a cyber attack commencing on 30 July, 2026."
Pokémon Center says unauthorized parties may have obtained customers' full names, mailing addresses, phone numbers, email addresses, and details about the contents of their PokemonCenter.com orders.
The company says other information related to customers and their orders was not impacted and that CEVA does not have access to customers' payment card details.
Pokémon Center is currently displaying a notice on its UK website warning that some orders are experiencing delays and may take longer than usual to process, dispatch, and deliver.
However, customers are also reporting that the breach caused their orders to be canceled, although it is unclear why the cyberattack would require cancellations rather than simply delays.
While initial reports warned of cancellations for the highly anticipated 30th anniversary collection products, a Reddit post shows that other merchandise, such as the Ghost Chateau Cyndaquil keyring, was affected.
Another customer replied that they had also received the same cancellation email.
BleepingComputer contacted Pokémon Center and Pokémon media contacts to learn more about the breach and why the incident caused customer orders to be canceled, but has not received a reply.
Overall prevention scores can hide what happens after initial access. Once attackers are using valid credentials, prevention drops sharply.
The Blue Report 2026 measures defenses technique by technique across 338 million simulations run in customer production environments.
Amazon’s team uses a T. rex preparing to devour a book as its logo.
For the past year or so, booksellers have suspected that AI firms are buying up huge lots of rare books, then destroying them after scanning them to train AI. But this was hard to prove until now, as 404 Media reports that an Airtag hidden in a rare book shows that at least one tech giant, in the race to advance its frontier models, is behind some of the bulk orders: Amazon.
On Monday, 404 Media revealed that it had connected with a bookseller who agreed to plant an Airtag in a rare book that was part of a bulk order. That Airtag was then tracked to an Amazon AI training facility in Las Vegas that housed a team focused on tearing books from their spines and scanning pages, 404 Media reported. Apparently tone-deaf to the escalating backlash over destructive book scanning, a logo on the door of that team’s warehouse, VGT3, showed a Tyrannosaurus rex preparing to devour a book, 404 Media documented.
Amazon declined to comment on 404 Media’s findings, only providing Ars with the same statement it gave to 404 Media, which does not mention AI training specifically.
“Amazon purchases books through commercial channels to help develop and improve the products and services our customers use,” Amazon’s statement said.
However, Amazon is developing what it considers frontier AI models, which require a massive amount of unique training data to stay competitive with leading firms like Google, OpenAI, or Anthropic. Right now, firms carefully guard their training data to avoid losing an edge. And training models on text from rare books that are difficult to find would seemingly offer an advantage for Amazon, especially since rivals like Anthropic and xAI have publicly stated that they are not training on rare or antique books.
Further, it seems that Amazon needed a new source of original text. 404 Media flagged discussions in online forums where VGT3 workers suggested that earlier this year Amazon had run low on books to scan. The shortage was so alarming that they worried the warehouse might shut down if Amazon couldn’t find more books to scan. At one point, the supply completely ran out, workers said. But the facility is still operational, 404 Media reported. And it’s now confirmed that bulk orders of rare books delivered there are systematically destroyed by this crew.
Many book lovers are horrified by destructive book-scanning (since there is an alternative) , with one staunch critic, Michael Burry, even reportedly labeling the practice to be “evil incarnate.” However, Amazon workers reported in forums that they consider the gig to be a “nice” opportunity for those drawn to a humdrum job with flexible hours.
On top of revealing that rare works that booksellers value are getting chewed up and swallowed by Amazon’s AI machine, 404 Media suggested that its investigation helped firm up another bookseller theory about why AI firms might be ordering certain rare books and not others.
After a reportedly historic year of sales, booksellers had suspected that AI firms were targeting books with ISBN numbers in order to ensure that the highest volume of unique works were present in training data sets. And 404 Media’s review of Amazon workers’ online discussions indicated that they were trained to scan barcodes or ISBNs before scanning books. That practice, 404 Media reported, “gives further credence” to booksellers’ theory that “AI companies are trying to methodically scan every printed book in the world by working through the list of ISBNs.”
For booksellers, the money may be good, but the risk that their carefully sourced collections will be destined for destructive book scanning like Amazon’s raises an ethical dilemma. They know how to assess a wide range of rare books to determine their value, and AI firms seem to be skipping that step in hunting low-cost, unique ISBNs to complete their checklists.
Right now, the books that AI firms are apparently buying up aren’t necessarily the kind of prized first editions of celebrated works that are typically valued quite highly. Instead, AI firms often target older books with lower monetary value, such as books that were never translated from a foreign language that’s not widely used today or books that were never popular enough to be widely distributed.
However, these works may still have “historical value, intellectual value, sentimental value” that AI firms overlooked, the bookseller who planted the Airtag told 404 Media. A rare book’s value can be derived from “all sorts of things” that “the AI companies don’t care about. They just want the content as a bunch of words strung together.”
On Reddit, some book fans debated whether it was that problematic that companies are destroying rare books to train AI, especially since, as the BBC reported , some of these books have been sitting on booksellers’ shelves for decades gathering dust.
“You were not going to buy that old paper book,” one Redditor commented in response to a post lamenting that “an obscure book from 1700 is now a museum piece and may reveal day-to-day stuff that we didn’t know.”
In that thread, the original poster said that the real problem was that tiny details and even major historical insights that can be gleaned from reviewing rare books will be lost to AI greed. AI firms will “never share the contents” of books they scan, the poster said, “as they don’t want anyone else to be able to train their AI” on the same works.
“This sounds like propaganda from the AI haters,” another Redditor pushed back, but a subsequent commenter shared similar fears. Although people might clash with someone who argues that training AI on these works will make all the knowledge that a work contains more accessible online, the commenter suggested instead that AI models would only make available “warped, censored, and paywalled fragments of ideas” from forgotten works.
“Even if you love the technology, you can admit that the concept of an AI literally eating books to become more powerful is pretty dystopian,” someone else on the thread said.
Some booksellers agree that not every text needs to be saved, the BBC reported. However, Scottish bookseller Derek Walker told the BBC that AI firms should be striving to distinguish between works that won’t be missed much—such as little-known academic texts that may technically be rare—and lesser-known antique works that may be “the only known surviving example of an edition.”
“It would be a much more significant problem if one like that were to be bought for destruction, having survived this long,” Walker said.
For AI firms guarding their training data and using services that mask their identities as buyers, there’s likely little desire to discuss their bulk buying directly with booksellers. Allowing booksellers to weigh in on the works they plan to feed into their models would require a level of transparency that may seem riskier than a possible reputation hit if it’s ever proven that a treasured first edition was destroyed in the name of advancing AI.
Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.
If you have read the source code of the
sync
package, you may have noticed that several structs contain an unusual field of type
noCopy
, such as
sync.Mutex
,
sync.Once
, and
sync.Map
:
go
type Mutex struct {
_ noCopy
...
}
type Once struct {
_ noCopy
...
}
type Map struct {
_ noCopy
...
}
noCopy
is a special marker for types that must not be copied after their first use. But the marker itself is only an empty struct with two empty methods:
go
type noCopy struct{}
func (*noCopy) Lock() {}
func (*noCopy) Unlock() {}
Despite the method names, there is no lock and nothing gets unlocked. But nothing here stops us from copying the value. This post explains what can break after a copy, why
noCopy
needs these two methods, and how to add the same marker to your own types.
The
noCopy
marker does not add any special rule to the Go compiler. You can still copy a
sync.Map
after it has been used:
go
var a sync.Map
a.Store("k", 1)
b := a // copying a sync.Map
The assignment copies all the fields from
a
into
b
, exactly as it would for any other struct value. The code still passes
go build
because the compiler gives no special meaning to the name
noCopy
or to its
Lock
and
Unlock
methods.
It turns out that the warning comes from a separate tool called
go vet
. This static-analysis command is included with Go and reports suspicious code that the compiler still accepts. When
go vet
checks the same assignment, it reports:
assignment copies lock value to b: sync.Map contains sync.noCopy
The phrase
copies lock value
comes from the purpose of the
copylocks
checker in
go vet
. It was created to report copies of values that contain a lock, such as
sync.Mutex
, because copying a lock after it has been used also copies its internal state.
The behavior is easy to reproduce with a regular struct that contains
sync.Mutex
:
go
type Counter struct {
mu sync.Mutex
value int
}
func main() {
var a Counter
b := a
_ = b.value
}$ go vet ./...
main.go:12:7: assignment copies lock value to b: example.com/nocopy-repro.Counter contains sync.Mutex
Counter
contains an actual mutex, so copying the outer struct also copies the mutex state.
So
go vet
produces this warning through its
copylocks
checker and this checker does not search for a field named
noCopy
. It uses the following rule while inspecting the copied type and its fields:
go
if types.Implements(types.NewPointer(typ), lockerType) &&
!types.Implements(typ, lockerType) {
return []string{typ.String()}
}
The checker starts with the type being copied (e.g.,
Counter
) and asks whether its pointer implements
sync.Locker
while its value does not. If the type is a struct and the answer is no, the checker applies the same rule to the type of every field, including fields inside nested structs. For
Counter
, this search finds
sync.Mutex
in the
mu
field.
For
sync.Once
, it finds
noCopy
in the
_ noCopy
field. This recursive search is why the warning can report that an outer struct contains a lock or
noCopy
.
That is why the
noCopy
marker’s methods are named
Lock
and
Unlock
. As mentioned, the rule above checks whether a type implements
sync.Locker
. This interface contains exactly two methods:
go
type Locker interface {
Lock()
Unlock()
}
type noCopy struct{}
func (*noCopy) Lock() {}
func (*noCopy) Unlock() {}
These names make the pointer
*noCopy
implement
sync.Locker
, while the value
noCopy
does not.
noCopy
was added to the standard library in 2016 by
Aliaksandr Valialkin
, CTO of
VictoriaMetrics
, based on a pattern proposed by
Russ Cox
. These types were already unsafe to copy. The change gave
go vet
a way to report copies it had previously missed. The 2016 implementation had only
Lock
. In
2018
,
noCopy
gained
Unlock
when
copylocks
switched to checking
sync.Locker
.
But you may notice that some structs in the
sync
package already contained a
sync.Mutex
, such as
sync.Map
:
go
type Map struct {
mu Mutex
...
}
If
go vet
could already find the mutex, why does
sync.Map
also need
_ noCopy
?
sync.Map
did not need
_ noCopy
merely to trigger a warning.
copylocks
could already reach its
mu Mutex
field. The explicit markers in
sync.Map
,
sync.Once
, and
sync.Mutex
have three purposes.
First,
_ noCopy
gives each struct an explicit marker, so
copylocks
does not depend on how that struct is implemented. The checker can find
_ noCopy
directly instead of finding an internal mutex or relying on the outer type’s
Lock
and
Unlock
methods.
Second, the warning describes the copying restriction more clearly. The old warning for
sync.Map
named the mutex found inside it:
assignment copies lock value to b: sync.Map contains sync.Mutex
The current warning names
noCopy
instead. The name directly indicates that the outer type must not be copied:
assignment copies lock value to b: sync.Map contains sync.noCopy
Third, the marker fixes a false negative involving
sync.Mutex
. A false negative means the checker should report a copy but does not. The checker recognized
sync.Mutex
through its
Lock
and
Unlock
methods, but a new named type does not inherit those methods:
go
type LocalMutex sync.Mutex
Before the explicit marker was added to
sync.Mutex
,
copylocks
could miss a copied
LocalMutex
. Its underlying struct now includes the
_ noCopy
field, so the recursive search can find the marker even though
LocalMutex
does not have the methods of
sync.Mutex
.
Note
Interestingly,
sync.RWMutex
is the only exported struct in the
sync
package that must not be copied after first use but does not contain a
noCopy
field. But
go vet
still catches a direct copy because
*RWMutex
implements
sync.Locker
while
RWMutex
does not.
What about a new defined type that loses those methods, just like
LocalMutex
?
go
type LocalRWMutex sync.RWMutexIts source provides another path for the checker:
go
type RWMutex struct {
w Mutex
writerSem uint32
readerSem uint32
readerCount atomic.Int32
readerWait atomic.Int32
}
The recursive search reaches the highlighted
w
field, so
copylocks
still reports
LocalRWMutex contains sync.Mutex
. The old
LocalMutex
had only integer fields after it lost the methods, which is why that type needed the explicit marker to fix its false negative.
But it’s an exception from a consistency POV.
Now that we understand how the marker works, your package can define its own version.
sync.noCopy
is unexported, so code outside the
sync
package cannot use it directly:
go
type noCopy struct{}
func (*noCopy) Lock() {}
func (*noCopy) Unlock() {}
type Session struct {
_ noCopy
id string
closed bool
}
The marker still cannot stop the compiler from copying a value.
go test
runs several vet checks by default, but
copylocks
is not one of them. A default
go test ./...
run therefore says nothing about this warning:
sh
$ go test ./...
ok example 0.4s
$ go vet ./...
lib.go:7:7: assignment copies lock value to b: example.Session contains example.noCopy
Many projects run
go vet
directly, either as a developer command or in CI, so
copylocks
can report copied values. If you prefer to run the check through
go test
, enable the
copylocks
checker explicitly, either by itself or as part of all vet checks:
sh
go test -vet=copylocks ./...
# Or run all vet checks.
go test -vet=all ./...
Both commands run
vet
on the package source and its test source files before running the tests.
There is no single failure behind this warning. Different types can behave differently after they are copied.
go
func finish(w sync.WaitGroup) { // w is a copy
w.Done()
}
var wg sync.WaitGroup
wg.Add(1)
finish(wg)
wg.Wait() // never returns
The call to
finish(wg)
copies the fields of
wg
into the parameter
w
, including the counter value of 1.
w.Done()
changes only the copied counter from 1 to 0.
The original counter stays at 1, so
wg.Wait()
has no reason to return. This is the classic case, and it is the reason many developers first meet the warning.
go
var a sync.Map
a.Store("x", 1)
b := a
b.Store("fromB", 1)
_, seen := a.Load("fromB") // true
The first
Store
initializes
a
and gives it a root pointer. The assignment to
b
copies that pointer, so
a
and
b
both access the same trie. If we later call
Store
on
b
, the new entry may also be visible through
a
, even though
a
and
b
are separate variables.
The shared storage of
aandbin this example comes from the root pointer in the currentsync.Mapimplementation. Since that copy happens after the map’s first use, its behavior is not guaranteed, and a future implementation may fail differently. We will discuss the current implementation in a dedicatedsync.Maparticle soon.
The behavior changes when we make the copy before the first
Store
:
go
var c sync.Map
d := c // copied before either is used
c.Store("fromC", 1)
d.Store("fromD", 1)
_, cSeesD := c.Load("fromD") // false
_, dSeesC := d.Load("fromC") // false
Because the copy happens before the first use,
c
has no root pointer to copy into
d
, a detail of the current
sync.Map
implementation. Both variables start empty, and each one initializes separate map storage when
Store
is called. This is why neither variable sees the entry added through the other.
Go’s documented
sync.Map
contract
allows this copy:
"A Map must not be copied after first use."
But
copylocks
still reports it as the checker does not track whether the map has already been used. Even so, copying a
sync.Map
is usually a bad idea since the code then relies on order-dependent behavior.
noCopy
has size zero, but a trailing
noCopy
field can still increase the total size of its containing struct because Go may add padding after it. A first-field position avoids that extra padding:
go
type plain struct {
n int64
}
type first struct {
_ noCopy
n int64
}
type last struct {
n int64
_ noCopy
}
Using
unsafe.Sizeof
on a 64-bit platform such as amd64 or arm64 gives:
plain 8 bytes
noCopy first 8 bytes
noCopy last 16 bytes
In
first
,
noCopy
has size zero and starts at offset 0. The
int64
can also start at offset 0 and spans bytes 0 through 7, so the whole struct uses 8 bytes.
In
last
, the
int64
spans bytes 0 through 7, which puts
noCopy
at offset 8. A struct with a size of 8 bytes ends at that same offset, so the address of
noCopy
would sit outside the struct’s allocated memory.
Go adds 1 byte after the field, then rounds the total size up to the struct’s alignment. On amd64 and arm64, the alignment is 8 bytes, so the total size increases from 8 to 16 bytes. On 386 and arm, the alignment is 4 bytes, so the same struct has a total size of 12 bytes.
The standard library avoids this size increase by putting
noCopy
before fields with a non-zero size. In
sync.Map
,
sync.Once
,
sync.Pool
, and
sync.WaitGroup
, it is the first field.
atomic.Pointer[T]has another zero-size field beforenoCopy:go
type Pointer[T any] struct { _ [0]*T _ noCopy v unsafe.Pointer }Both fields above
vhave zero size. The pointer that stores the state still comes afternoCopy.
In Patagonia, los puesteros look after other people’s livestock for not very much money, and sometimes go months without seeing anyone else.
The landscape in the southernmost part of Chilean Patagonia, often called the End of the World, has a post-apocalyptic quality. At the final stop, so to speak, before Antarctica, the weather is harshly volatile, and in the wilderness surrounding the few small cities and towns, civilization dwindles almost to nothing. Most of the human presence in that region is transient—tourists who marvel at nature and soon depart. But a small number of people do live on the land, in isolated cabins, often going weeks, and sometimes months, without seeing one another, or anyone else. Known as puesteros , they are solitary men employed by landowners to look after ranches and livestock. Because this onerous and not very lucrative work is unappealing to younger generations, the puesteros are an aging breed, with few lined up to replace them.
In 2018, during a nine-month stay in Patagonia, the Dutch photographer Pie Aerts often spotted puesteros from a distance: lone figures atop horses, silhouetted against the horizon. “In that huge landscape, they appeared stoic, but extremely vulnerable, as well,” Aerts told me recently. Those distant encounters were the whole of his contact with the men during that trip; it wasn’t until the compulsory lockdown of 2020, which Aerts endured back home, in Amsterdam, that he began to consider the effects of prolonged isolation. Recalling his visions of puesteros , he searched for more information online and came across a short documentary that chronicled their seclusion. He reached out to the film’s director, Matías Bolla, an Australian filmmaker of Chilean descent who was also spending quarantine in his home country. The exchange led the pair back to Chile in 2022, on the first of several trips that have resulted in a feature-length film, currently in post-production, and the photo book “ Coirón ” (GOST Books), which takes its title from an unusually resilient grass that is native to the region.
The book features portraits of puesteros between shots of domestic settings and local landscapes—the little that belongs to them and the vastness that does not. Though Aerts initially intended to focus on solitude, he came to see that the lives of these men were equally shaped by precarity and dispossession, by “the fear of retirement that they all carry, with few pension funds or social-welfare conditions for them in place,” Aerts said. He quoted a puestero who said, “What a shame to come to love the land that will never be mine.” In one photo from the book, two paltry berries rest on a man’s open palm, fresh soil under his fingernails.
Outside the modest cabins, seasons pass. One stands starkly in the dead white of winter; another appears almost to melt into the pink-hued mountains surrounding it. Inside the homes, sunlight falls on faded posters, worn furniture, and an old radio by a window. Other photos present less legible scenarios. There is a taxidermied puma lying in the middle of an abandoned room, five taxidermied armadillos arranged on a couch, three mattresses collapsed under an overcast sky. These images baffle, then linger, suggesting that life in this remote countryside is stranger than we may understand it to be.
Aerts is not fluent in Spanish; during visits with the men, Bolla conducted interviews while Aerts sat in the background, absorbing what he could. This turned out to be a fertile constraint. His relationships with the puesteros were largely built on nonverbal exchanges: playing music, cooking dinner, sharing in stillness. “Sometimes it was required to sit in a room and not speak at all, just listening to the wind,” Aerts said. “It was a big shock for me, coming from a world where everything is so connected, and where we’re always filling the void.” Silence is palpable in the portraits of the men, photographs that show hardship coexisting with resilience, that quiver with pathos but do not collapse.
A scientist with a Ph.D. from Harvard fatally shot three of her colleagues. Then revelations about her family history came to light .
Roboflow Playground lets you run the same image and prompt across up to five zero-shot computer vision models side by side, covering more than 30 models from Anthropic, OpenAI, Meta, Google, and open-source providers like Florence-2. Supported tasks include object detection, image classification, OCR, captioning, and open-prompt visual question answering. It removes the setup cost of provisioning APIs or infrastructure individually, so you can compare model outputs directly before committing to one for your project.
Comparing zero-shot computer vision models – from Claude Opus 4.7 to Gemini 3.1 Pro – can be daunting. Researching the latest models to try, writing the code to call cloud APIs, provisioning infrastructure for open weight models – all of this takes time. Before you know it, a new model is out, ready for use.
With that in mind, we are excited to announce a tool to help you try, compare, and evaluate over 30 popular zero-shot vision models: Roboflow Playground .
In this blog post, we are going to walk through what Roboflow Playground is, and how to use it to test new vision models.
Roboflow Playground lets you compare, side-by-side, the latest computer vision models.
With Playground, you can run the same image and prompt across up to five vision models at once. Supported models range from the latest VLMs by Anthropic, Meta, OpenAI, and Google, all the way to open source models like Florence-2 and YOLO World.
To get started, go to the Roboflow Playground website. You will then be able to choose what vision task you want to run. As of today, Playground supports:
You can then choose up to five models to compare. Once you have chosen a task and a model, you can upload an image and set prompts.
The models available depend on the chosen task type. For example, you can use Florence-2 and Seg Preview for object detection because both models support object detection, but you can't use these models for VQA because they don't support this task.
Let’s try Roboflow Playground on object detection.
To choose a task, click “Object detection” from the list of tasks dropdown and select your chosen task:
We can then upload an image of a book and coffee and set the prompts “book” and “coffee”:
We are now ready to run our image and prompt through vision models to see the results. For object detection, Playground automatically plots the bounding boxes returned by each model.
Here is an example of the results for our prompt:
In this example, we ran our object detection prompts – “coffee” and “book – through Florence-2 , YOLO World, and Claude 3.5 Sonnet. Both Florence-2 and YOLO World identified both objects and drew accurate bounding boxes; Claude 3.5 Sonnet found the general location of each object, but was unable to draw precise bounding boxes.
Let’s try using Playground with the Open Prompt task type. This lets us ask a question about the contents of an image. To use Open Prompt, choose “Open Prompt” from the task dropdown in the top left corner of the Playground interface, then upload an image and set a question to ask.
For this guide, let’s upload a picture of a cup of coffee on a table and ask “What is in this photo?”
Here is an example result from the Playground:
Above, results from Claude 4 Sonnet and GPT-4.1 are displayed. Both models accurately identify that the photo contains a coffee cup on a table, and include detailed descriptions of the background and surroundings.
With Roboflow Playground, you can try various computer vision models and compare them side-by-side.
Today, we are launching with support for 30+ models, including the latest models by Anthropic, OpenAI, Google, Mistral, and other providers, as well as open-weights models like the latest in the Llama series.
We plan to add more models as they become available.
You can get started with Roboflow Playground today to try, compare, and evaluate supported vision models for free.
If you come up with a prompt that returns interesting results, share it on social media. You can tag us @Roboflow to let us know what you have found when playing with the latest frontier vision models.
My book has a
linter
that yells at me for hyphenating “open source.”
1
It runs ~5,500 automated checks, rebuilds five formats on every
git push
, and fails the build if I so much as imply I still work at a job I left. For a book. That one person wrote.
I didn’t set out to do this. Most people start a writing project by firing up Word or Google Docs, and I started down that same path. I even tried some purpose-built authoring tools, but every tool felt inferior to the ones I used as a developer every day. I did “the only reasonable” thing and threw all of them out, writing the whole book on Git, Markdown, and a CI build pipeline. If you’ve read how I over-engineered my home network — twice — none of this will surprise you.
I’ve been making websites for decades, so the leap was short: the same tools that build websites could build books, no last-minute magic-trick reveal required to abracadabra a standard Word doc into a full-blown book. There were missteps, 2 but in the end, I would not have written Open and Async any other way. Here’s how:
Naturally
, the content itself lived as Markdown files in a Git repository. After all, that’s where I spend most of my day. I used VS Code, with a handful of prose extensions (
listed below
). Each chapter was its own Markdown file, and a single
index.yml
file defined the order, making it easy to re-order chapters or add new ones.
Practically, I wrote most of this book on an iPad — Codespaces in a browser tab, a Bluetooth keyboard, often nights and weekends while away from my desk — and the Git repository kept everything in sync no matter where I opened it. I could focus on the words, and a bad idea was one
git revert
away from gone.
Not to mention, I had real-time feedback on my writing right in my IDE from the various prose linters, just as I would have real-time feedback on my code from ESLint or Prettier.
With content as code, the next logical step — and the point where “reasonable” quietly left the building — was to set up automated tests. Testing prose the way you test code is something I’d argued for years; this was me taking it to an absurd extreme. I did that two ways: real-time, and on push (CI).
Locally, as I typed, I ran several VS Code extensions all giving me real-time feedback. Specifically:
All six also ran in CI (Alex and Write-good folded into Vale there; more on that below ). Together, they layered hundreds of curated style rules over a full grammar engine — all of it underlining my mistakes in real time, the way a red squiggle flags a type error.
In addition to running those open source linters in CI (some blocking), I built a custom test suite of my own: a standalone Node script of content validators, a Vitest suite, and Playwright specs. The validators are the fun part. Each is a few lines that read the Markdown and push an error with a file-and-line pointer. My favorite: I don’t work at GitHub anymore, so any sentence claiming I still do fails the build.
// My time at GitHub has to read as past tense — present-tense
// employment claims about it fail the build.
function validateGitHubTense(files) {
const patterns = [
// "is/are ... at GitHub" — a current-employment claim
/\b(is|are)\b[^.!?\n]{1,80}?\bat GitHub\b/i,
// "works/leads/runs at GitHub" — current-employment activity
/\b(works?|leads?|runs?|manages?|directs?)\s+(at|for)\s+GitHub\b/i,
];
return flagLinesMatching(files, patterns);
}
That validator exists because I made the mistake once — which is the whole pattern. The first time an error got past me, I didn’t just fix that one sentence; I wrote a rule so I’d never have to catch it by eye again. It’s cattle, not pets for prose: don’t hand-nurse each chapter, govern the whole herd with policy. A mistake caught once becomes a check that sweeps every chapter and fails the build if it ever wanders back. That’s one of about thirty. Others I’m proud of:
validateOpenSourceHyphenation
— “open source” is
a noun, not a verb
, and never hyphenated.
validateHypotheticalHooks
— formulaic AI-tell openers (“Picture this…”, “Imagine…”, “Consider a…”) at the start of a paragraph.
validateSentenceStarters
— three-plus sentences in a row opening with the same word.
validateNoBareUrlLinkText
— no link whose visible text is just the raw URL.
validateCalloutBalance
— the book speaks to managers and individual contributors, so a “For managers” callout has to have a “For
ICs
” counterpart nearby.
validateCrossReferences
— every
[text](#anchor)
cross-reference resolves to a real heading.
…plus a couple dozen more for small caps, em-dashes, doubled words, en-dash ranges, TL;DR length, and every other tic I could name. 5
One of these caught
me
.
validateHypotheticalHooks
flagged the opening of a paragraph I was certain I’d written myself — and I had. But reading it back cold, it did sound ghost-authored; I’d absorbed the cadence from reading too much generated text and produced a fluent imitation of nothing. A linter can’t tell good prose from bad. What it can flag are the patterns you reach for when you’ve stopped thinking — which is exactly the thing you can’t see in your own draft. Same reason
eslint
earns its keep: it can’t tell good code from bad either, but it catches the autopilot mistakes your own eye skates right over.
All in all, the CI suite ran ~70 test files with 2,204 test cases and ~3,900
expect()
assertions — plus another ~1,600 per-chapter structural checks from the validators above.
The linters caught mistakes, but they couldn’t tell me whether the book repeated itself or whether a chapter was any good. For that I built a second layer of tooling: audits that judge the writing, not just check it. One thing they never did, though, was write it. Every word is mine; these tools are readers of last resort — catching what I’d stopped being able to see.
After reading the book over and over, I was convinced I’d repeated the same idea across chapters. I wanted proof, not a hunch — so I built three layers of duplication detection, each catching what the one before it misses:
jscpd
— token-level copy-paste detection. Catches longer verbatim blocks I’d pasted between chapters, but nothing subtler.
6
index.yml
, strips Markdown/Pandoc syntax, builds word n-grams (phrases of n words), and flags phrases appearing in more than one chapter (plus a “most-repeated stock wording” ranking). Two modes: cross-chapter (default
--n
) and intra-chapter (
--scope=intra --n=8
) for a chapter repeating itself.
cross
misses.
Was I actually repeating myself? By this final pass the two mechanical layers came up clean — but they’d earned it:
jscpd
and the n-gram scan had already caught the copy-paste and recycled phrasing (a doubled motif here, a reused TL;DR structure there), and I’d fixed each.
What neither could see was the subtler kind — nothing was duplicated word-for-word anymore, just the same point in different words. The semantic pass caught that, and it was right: I’d made the same argument, that moving office habits online isn’t the same as working remote-first, in five separate chapters, on top of 166 smaller self-restatements scattered across 51 chapters. I’d paraphrased myself too well for anything cheaper than an LLM to catch me. (share this quote)
My hunch was correct; I’d just needed three escalating tools to prove what re-reading my own book to blurry-eyed exhaustion couldn’t.
Beyond duplication, a second set of tools graded the prose itself — split by how they judge. Some are probabilistic (an LLM reads the chapter and forms an opinion; run it twice and the findings can shift), and some are deterministic (rules and arithmetic — same input, same output, every time).
The Claims lens paid for itself early. It stopped on a sentence claiming GitHub’s monthly all-hands “dedicated roughly half of each session to live Q&A” — a number I was certain of and had wrong; it was closer to a third, and the share moved around over the years. No linter flags that: it’s clean, confident prose that happens to be false. Only a reader asking “is this actually true?” catches it, and I’d have shipped it otherwise.
Claims is one of
20 single-purpose lenses
in a “prose audits” test — a per-chapter LLM auditor where each lens asks one narrow question so the model can’t hand-wave a vague “looks good.”
--lens=all
runs every lens;
--models
/
--rounds
add a deduplicated multi-model and self-consistency panel, so a finding has to survive more than one model (or more than one run) to count. A sampling of the rest:
7
| Lens | What it catches |
|---|---|
| AI-tells | Phrasing that reads as machine-generated rather than my voice |
| Hook | Whether the opening earns its place or is throat-clearing |
| Legal | Legal or reputational risk (naming names, unverified claims) |
| Dated | Perishable references that will age badly (“recently,” “this year”) |
| Global | Idioms and cultural assumptions that trip up non-US readers |
| Dual-audience | Whether a chapter serves managers and individual contributors |
| Promise | Whether the chapter delivers on the book’s core promise |
Separately, I also had a traits analysis test, which scored each chapter on persuasion and engagement traits, catching prose that was technically clean but flat so I could give it another (human) pass.
I ran these after a round of editing, to see whether the book was improving as a reading experience: chapter- and paragraph-length distributions, whole-book reading time, a hard
EPUB
size budget (Kindle penalizes oversized files on delivery), a book-wide consistency checker, and a stats dashboard whose
--check
mode fails the build unless every “attention item” — a chapter missing a TL;DR, an unbalanced callout — sits at zero.
8
Together, these turned “is the book actually getting better?” from a gut feeling into a number I could watch move between drafts.
I am far from a designer, and I’d never published an ebook before — I had no idea how they worked beyond reading many of them. Two aha moments changed the way I thought about publishing:
@media print
and
@page
rules, including left and right page styling, title pages, page numbers, and more.
For the interior design, I reached for
Tailwind CSS
— not because it’s built for books (it isn’t), but because it’s what I already use every day.
@tailwindcss/typography
gave me a typographic baseline to start from instead of a blank page.
One note: I purposefully hired a human designer for the cover. For a book about being authentic, the first impression had to be authentic. 10
Styling every format from one stylesheet has a failure mode: CSS meant for one format bleeding into another. To keep my e-reader tweaks away from the paperback, I scoped each of them to
@media not screen
. But
WeasyPrint
— the HTML/CSS-to-PDF engine that draws the paperback,
more on it below
— renders the print PDF with media type
print
, and
print
is
“not screen,” exactly as the spec says. So two of those e-reader rules matched anyway and quietly bled into the book: a tighter line-height on callout paragraphs, and a
box-decoration-break
change that dropped the repeated padding on any callout that split across a page.
Neither was visible. Nothing
looked
broken. But each shaved a sliver of vertical space off every callout, and the book slowly deflated from 576 printed pages to 567. I only found out because the print PDF is pinned to exactly 576 pages
11
and the build went red — a page-count gate written to protect the cover spine, catching a CSS bug it was never designed for.
Two formats from one stylesheet is a gift, right up until the cascade forgets which format it’s in.
(share this quote)
The fix re-pins the original values in a print-only block, loud with
!important
so the leak can’t win.
The design and layout got a test suite as well — driven by Playwright against the built HTML, so it checks what actually renders, not what I hoped the CSS did. Two categories:
The
axe-core
engine runs over the rendered book in
both light and dark mode
against
WCAG
2.1 AA — no critical or serious violations, AA color contrast in either scheme, alt text on every image, logical heading order, discernible link text (no bare “click here”), and a declared
lang
attribute so screen readers pick the right pronunciation. It’s the
test:a11y
gate, enforced in CI as
validate-playwright
. Accessibility earned more than a subsection here — it has
its own accessibility statement
, and a dedicated deep dive is coming.
These assert that the actual computed styles match the design intent — the details that break quietly and never show up in a diff. A representative few: the title page renders centered with a lighter-weight subtitle; the table of contents is a real
nav
with the
doc-toc
role and working links; every callout type carries its own unique left-border color and auto-generated label prefix; and small caps get real
font-variant: small-caps
, not just a class name.
12
None of this is glamorous. It’s exactly the kind of thing that breaks silently in a Friday CSS refactor and doesn’t surface until a reader emails you.
We’ve got clean Markdown, and a test suite that ensures the content is clean, but we still need to get it into a format that can be published (EPUB, Kindle, paperback in two flavors, and web). Core to that custom build pipeline was Pandoc , a universal document converter that can read Markdown and spit out just about anything else. I used Pandoc as the engine, but Pandoc alone gets you a generic document — turning that into five store-ready formats took a pile of custom tooling:
Pandoc is only the engine. The full build also needs WeasyPrint to draw the PDF, Ghostscript to convert it, and a pile of Noto fonts for full Unicode coverage — a gnarly stack of native dependencies to install by hand. The whole thing therefore lives in a Docker image and a dev container, pinned to an exact renderer version (a detail that matters more than you’d think — see the page-count gate below ). That container is also why I could draft the book from an iPad: the heavy toolchain ran in Codespaces, not on my lap.
Everything starts on a throwaway copy of the source tree, so these transforms never touch the real chapters. A couple of scripts massage that copy before Pandoc runs — stamping each build with its commit SHA, turning inline links into numbered footnotes for print (a hyperlink is useless on paper; a footnote with the full URL isn’t), and reshuffling the chapters per format so the EPUB gets shareable quote links while print and Kindle get a QR share page instead. 13
One Tailwind stylesheet (
src/style.css
) compiles through PostCSS to
dist/style.css
, so screen, print, and EPUB all start from the same source of truth. Each format then takes a different slice:
strip-page-rules.js
derives a cut-down stylesheet for EPUB, stripping the CSS Paged-Media features (
@page
,
target-counter()
,
oklch()
, custom properties) that Kindle and EPUBCheck choke on.
This is the part I didn’t expect to love. Pandoc parses everything into an abstract syntax tree (AST) and lets you rewrite that tree with small Lua scripts before it renders — so format-specific tweaks live in code, not smeared through the prose. Deleting a node, for instance, is just returning an empty table:
-- strip-comments.lua: drop editorial HTML comments so my notes-to-self
-- never ship inside the published .xhtml. Returning {} removes the node.
function RawBlock(el)
if is_html_comment(el.format, el.text) then
return {}
end
end
The rest are variations on that idea:
add-div-titles.lua
— injects the callout labels (“TL;DR: ”, ”
💡
Pro-Tip: ”, ”
👔
For managers: ”) and DPUB-ARIA accessibility roles, so those labels aren’t hardcoded in every chapter and can differ per format.
strip-emoji-kindle.lua
swaps them for Kindle-safe glyphs (e-ink has no emoji font) while
body-emoji-images.lua
turns them into inline Twemoji images for the print PDF.
tagline-share-links.lua
— appends a “share this idea” permalink after each of the book’s bumper-sticker lines in the EPUB.
Turning emoji into images solved the missing-font problem and created a subtler one. Twemoji’s PNGs are transparent, but the RGB underneath the transparent pixels is dark slate — invisible if a renderer honors the alpha channel, an ugly charcoal box if it doesn’t. And whether a renderer honors alpha turned out to depend entirely on where the book got opened.
On Kindle, my first fix baked the callout’s background color into each PNG and dropped the alpha outright. It looked perfect on a light-mode Paperwhite and terrible on Kindle for iOS, which does honor transparency and doesn’t paint the callout tint behind the image — so every emoji got a colored rectangle instead of a dark one. I’d traded one box for another. What shipped keeps the alpha intact but rewrites the RGB of every fully transparent pixel to white: renderers that honor alpha get clean transparency; the ones that flatten it get white, which vanishes on a white page.
The print PDF needed the
opposite
fix. Ghostscript converts the paperback to PDF/X-1a with
-dNOTRANSPARENCY
— required to keep the text as selectable vectors instead of a 300-DPI bitmap — and that flag
always
drops the alpha. So there, transparency is the enemy: each emoji is flattened onto the exact color it sits on (the callout’s gray, or white for body text) with no alpha at all, so there’s nothing to drop and no box to reveal.
The same bug needed opposite fixes — one runtime might drop the alpha, the other was guaranteed to.
(share this quote)
A Python script bakes both sets at build time.
It’s the most ordinary bug in the world — mine just happened to be in a book. 14
With the tree transformed, each format renders down its own path — and the paperback takes the longest road. HTML and the PDF both render through Pandoc, but the PDF is drawn by
WeasyPrint
, an HTML/CSS-to-PDF engine — so I typeset the entire book with the same box model I’d use for a web page. The EPUB goes a different way: Pandoc embeds WOFF2 font subsets, then
postprocess-epub.js
repackages it to stay valid and small. And the paperback keeps going after everyone else has stopped, out to Ghostscript to become
PDF/X-1a:2001
—
CMYK
color, an embedded USWebCoatedSWOP ICC profile, flattened transparency — the archaic print standard IngramSpark (the print-on-demand distributor) won’t live without.
That same 576-page gate earns its keep here too.
check-pdf-page-count.js
pins the count and fails the build the moment the paperback drifts off 576 pages — cheap insurance against a stale page count shipping as a wrongly-sized cover spine.
That’s five publishable files out of one Markdown source 15 — which raises the obvious question: how do I know any of them are actually correct?
The prose linters keep the words honest; a larger slice of that Vitest suite keeps the build honest. A Pandoc upgrade or a stray line of CSS can silently break a format, and I wouldn’t find out until a reader’s Kindle rendered the wrong font. The tests check the machinery: that each transform does what it claims, and — the biggest cluster by far — that typography hasn’t quietly drifted across small caps, table borders, print contrast, dark mode, and the e-ink emoji fallback. Each is a test because each is a way I’ve watched a build look fine and render wrong.
But unit tests only prove my code is right, not that the output is valid — so each artifact also runs through the same validators the stores themselves use, and a rejection happens on my laptop, not on upload day. Every EPUB goes through EPUBCheck , the validator every store runs on upload, and Ace by DAISY , an accessibility audit built specifically for ebooks; the built HTML gets crawled by lychee for link rot and run through an HTML validator before malformed markup can become a malformed ebook.
None of this is something I have to remember to run. Every push to
main
fans out into
14 parallel CI jobs
— build the CSS, then the EPUB, Kindle EPUB, print PDF, PDF/X-1a, HTML, and DOCX side by side, and validate each as its artifact comes ready. A green check means every format built and passed every gate; a red X means something’s off before I’ve even alt-tabbed away.
# build.yml — mostly independent jobs, so CI runs them in parallel
jobs:
build-epub: # → dist/book.epub
build-kindle: # → Kindle-specific EPUB
build-pdf: # → 6"×9" paperback PDF
build-pdf-ingramspark: # → PDF/X-1a for IngramSpark
build-html: # → web preview
validate-epubcheck: { needs: build-epub }
validate-ace: { needs: build-epub }
validate-playwright: { needs: build-html }
And because every build stamps its commit onto the title page, the book is versioned like software. I’m on release 1.0.1 — with a tag and a changelog, not a graveyard of
final_v3_revised_ACTUALLY_FINAL.docx
files. A typo fix is a point release.
Here’s the irony: after automating everything up to this point, the actual publishing is almost entirely click-ops. There’s no
git push
to production. Each store — Amazon’s
KDP
, IngramSpark for bookstores and libraries, Draft2Digital for Apple Books and Kobo — wants you to log into a web dashboard, upload the EPUB and the print PDF by hand, and re-enter the same metadata (title, description,
BISAC
categories, keywords, price) into a slightly different form each time.
My pipeline builds a flawless artifact, and then I upload it like it’s 2009.
(share this quote)
A few things kept even that part honest:
The nice thing about a book that builds like software: the same machinery keeps paying off after launch.
One Markdown source, a linter with opinions, and a build that yells when I’m wrong — here’s what all that over-engineering added up to.
| Metric | Count |
|---|---|
| Words | ~100,000 across ~70 chapters and sections |
| Print length | 576 pages (6”×9”) |
| Published formats | 5 (plus a DOCX I don’t ship) |
| Tests | ~70 files, 2,204 cases, ~5,500 automated checks |
| CI jobs per push | 14, in parallel |
| Style rules | ~300 covering 500+ banned terms, over a 5,000-rule grammar engine |
| Commits | 5,000+ to get here |
| Books | 1 (frankly over-engineered, but fun) |
The tooling outgrew the book, too. I pointed the whole apparatus at this blog’s fifteen-year archive and caught a 2011 post that had been saying “its clear” where it meant “it’s clear” the entire time — the exact missing apostrophe my linter now flags on every keystroke. Thousands of people read it; nobody ever mentioned it. You don’t find out. You have to go looking. (share this quote)
And it wasn’t only the tooling that transferred — the habits did. The instincts I sharpened here — treat a mistake caught once as a rule that catches it forever, keep a human in the loop on anything a machine drafts, make the build the source of truth — have already found their way into how I run my other projects, this site included. 19
I’m not going to tell you to write 5,500 tests for your novel. For most books, most of this is wildly disproportionate — and that’s the point. I did it because the marginal cost of one more check, one more format, one more validator was a few minutes and a little curiosity, and because doing it the developer way meant I could write from an iPad away from my desk and trust that
main
was always shippable. If you spend your days shipping software, the tools you already know will take you further than you’d expect.
The publishing industry starts with a word processor. I started with a Git repository
(share this quote)
— and I’d start there again.
Out now
Open & Async
The collaborative software development playbook for remote and distributed teams. Drawing on a decade at GitHub — on Kindle, Apple Books, Kobo & more.
Buy it — $9.99
What would you over-engineer, if you pointed your own tools at it?
A colleague called me out on “open-source” in an early draft; I vowed never to make the mistake again, then made a linter swear to it for me. ↩
Chief among them: I lost a weekend to LaTeX , on the theory that the tool serious typesetters swear by had to be worth learning. It is — if you typeset for a living. For me it was a wall of arcane macros and inscrutable errors to approximate a layout CSS gave me in an afternoon. I bailed and never looked back, which is how the whole book ended up drawn with a web box model instead. ↩
I’m a terrible speller so I also ran CSpell in VS Code, which shared a custom dictionary with Harper. ↩
Self-hosted, so it ran on-device too — well, in the Codespace. ↩
The roster, grouped.
Typography:
validateSmallcaps
and
validateEscapedSmallcaps
(acronyms rendered in small caps, not accidentally escaped),
validateEmDashes
,
validateNumberRangeDashes
(en-dash ranges),
validateQuotePunctuation
,
validateListPunctuation
,
validateBoldKeywords
.
Structure:
validateHeadingHierarchy
(no skipped levels),
validateDuplicateHeadings
,
validateParagraphLength
(no walls of text),
validateMinOpenerLength
,
validateSectionIntros
/
validateSectionIntroHasLead
/
validateSectionIntroFinalLead
/
validateSectionIntroNoTldr
(section intros lead into the content without restating the TL;DR),
validateTldrLength
,
validateChapter
,
validateIndexIntegrity
(
index.yml
matches the files on disk).
Links & references:
validateCrossReferences
,
validateNoBareUrlLinkText
,
validateFootnotes
,
validateAcronymExpansion
(expanded on first use).
Callouts:
validateCalloutBalance
(managers paired with ICs),
validateCalloutDivs
,
validateRoleCalloutLabels
.
Voice & anti-AI:
validateGitHubTense
,
validateOpenSourceHyphenation
,
validateHypotheticalHooks
,
validateSentenceStarters
,
validateDoubledWords
.
Housekeeping:
validateNoTodoMarkers
,
validateNoTransitionTodos
,
validateBuildIdentifiers
,
validateSSML
(audiobook markup).
↩
jscpd is not prose-specific. It’s a great way to DRY up code. ↩
The other twelve, briefly: Consistency (drift in the book’s own conventions), Taglines (bumper-sticker lines worth pulling into callouts), Proofread (typos and mechanical slips), Cross-reference (broken “see chapter X” pointers), Acronyms (unexpanded on first use), Jargon (the “explain it to a new hire” test), Alt-text (missing or non-descriptive), Evidence (assertions with no why or how), Structure (heading hierarchy), Readability (sentences hard to parse on one read), Inclusive (non-inclusive language, judged in context), and Emphasis (overused bold, italics, and scare quotes). ↩
audit-chapter-lengths.js
/
audit-paragraph-lengths.js
flag outlier chapters and walls of text;
reading-time.js
clocks each chapter and the whole book at 238 wpm (average adult non-fiction speed);
check-epub-size.js
enforces the size budget;
consistency-check.js
settles “open source” vs “open-source” and “async” vs “asynchronous” book-wide; and
writing-dashboard.js
renders it all to HTML. Plus a handful of advisory local checkers —
proselint
, GNU
diction
, and GNU
style
— for wordiness and readability stats.
↩
Not just a metaphor. Kindle’s modern format (KF8/AZW3) is rendered by a WebKit-based engine — the same rendering lineage as Safari — down to Amazon’s own
-webkit-
CSS extensions; a Kindle book really is a little offline web page. But the engine ages slowly across a long tail of e-ink models still in readers’ hands, so the HTML and CSS it reliably supports lag years behind the modern web — hence “very stripped-down.”
↩
AI could have generated something competent, and that was exactly the problem — competent and generic is the failure mode I spent 5,500 checks scrubbing out of the text . The cover is the one thing a reader judges before reading a word, so it was the last place I wanted a machine’s default taste. A human designer got the brief; the algorithm didn’t. ↩
Locking the length isn’t fussiness — the whole physical book is built around that number. A page count sets the book’s thickness, so the spine width, and the cover art wrapped around it, is computed from it. Drift off 576 pages and the cover no longer fits the spine: fixing it means a round trip back to the human designer to re-render the wrap, not a one-line code change. More on that gate below . ↩
The rest, in the same spirit: the copyright page is present and bottom-aligned on screen; H2 is larger than body text and H3 smaller than H2; links are underlined, code blocks have a dark background and inline code a visible one, and blockquotes carry a left border; and body text uses the Tailwind Typography
prose
class.
↩
update-revision.js
stamps the build SHA and version onto the title page, so every copy traces to its commit;
links-to-footnotes.js
does the print link-to-footnote rewrite; and a per-format manifest handles the reshuffle. Same source, subtly different books.
↩
I spent years at GitHub, where emoji are practically load-bearing — reactions,
:shipit:
, an emoji picker in every comment box — and they turned up a fresh encoding edge case like clockwork. Emoji, uh, find a way.
↩
The build actually emits a sixth artifact — a DOCX — which I used to gather early feedback from reviewers in Word, with track changes and comments. It’s a working format, not something I publish. ↩
The pipeline drafts and self-checks, but it doesn’t get the final word. If I decide to actually publish any of these, a human native speaker will review and proof the translation first — machine-assisted, human-approved. ↩
The QA suite catches drift, but a transcript diff can’t hear a flat line reading or a subtly wrong emphasis. If I release it, I’ll listen to the finished narration end to end — every word ear-checked by a human before it ships. ↩
The full list, with thanks to their maintainers: Tailwind CSS and Tailwind Typography , Pandoc , WeasyPrint , Ghostscript , qpdf , Poppler , Source Serif 4 , Source Sans 3 , and Source Code Pro by Adobe, Twemoji , ImageMagick , sharp , SVGO , PostCSS and cssnano , Remark and Retext (and countless ecosystem plugins), textlint , markdownlint , Prettier , Harper , Vitest , Playwright and axe-core , Ace by DAISY , EPUBCheck , htmlproofer , lychee , html-validate , Pillow , LanguageTool , Vale , proselint , GNU Diction , yamllint , js-yaml , jscpd , and just . ↩
Case in point: my résumé is a print-only web page the build renders straight to PDF — a different engine than the book (headless Chrome here, WeasyPrint there), but the same “ an ebook is a website in a trench coat ” idea, pointed at paper. ↩
In Brief
Posted:
Amazon is buying tons of rare books, cutting off their spines, and scanning them for AI training, according to 404 Media , which placed a tracking device in a rare book that ultimately arrived at an Amazon facility in Las Vegas.
The facility, known as VGT3, identifies itself with a symbol of a dinosaur holding a book in its claws. Amazon told 404 Media in a statement that it “purchases books through commercial channels to improve the products and services customers use.”
Companies like Amazon need unfathomably large amounts of text to train their LLMs, which have already ingested what they can from the internet (and, in Anthropic’s case, illegally pirated books). Rare books, especially ones that are out of print or impossible to find on the internet, offer a new source of coveted training data.
These texts are especially valuable since there’s no chance that anything published before 2022 was written by an LLM. When LLMs train on AI-generated text, they risk “ model collapse ,” which can occur when the quality of an LLM’s outputs degrade after ingesting too much AI-generated text.
Subscribe for the industry’s biggest tech news
Mark Zuckerberg, CEO of Meta, is seen in the U.S. Capitol after a meeting in the office of Senate Majority Leader John Thune, R-S.D., March 26, 2026.
Tom Williams | CQ-Roll Call, Inc. | Getty Images
If New Mexico created the blueprint for taking on Meta , California could determine the company's fate when it comes to critical changes at Facebook and Instagram .
Opening arguments begin Tuesday in the trial that California Attorney General Rob Bonta is co-leading against Meta, following a litany of allegations that the company fostered addictive behavior in teens and children. The jury was seated last week in Oakland's federal courthouse.
The trial involves a coalition of 29 state attorneys general in a unified case against Meta that was brought in 2023 , and will be argued by lawyers representing California, Colorado, New Jersey and Kentucky. The stakes are enormous as leading government officials across the country push for Meta to be held accountable for allegedly violating federal and state laws, including the Children's Online Privacy Protection Act, or COPPA, and various consumer protection statutes.
Industry experts are calling it social media's "Big Tobacco" moment , with Meta as the centerpiece. In the 1990s, tobacco companies were forced to pay billions of dollars for misleading the public about the safety and potential harms of their products, and subsequently saw their power and influence dramatically diminished.
Earlier this month, Meta lost a case in New Mexico that will require the company to make some changes to its services and pay nearly $1 billion. But California carries such power and influence that a bad result in Meta's home state could lead to much steeper penalties, including a broader overhaul that forces fundamental alterations to key algorithms.
"California matters more than any other jurisdiction in the U.S.," said Julia Powles, executive director of the UCLA Institute for Technology, Law and Policy. "It's where they are subject to the greatest legal reach, and it's a jurisdiction watched around the world."
New Mexico AG Raúl Torrez, fresh off his court victory, told CNBC that the consequences could be "astronomical" for a company that gets 98% of its revenue from online advertising. Meta CEO Mark Zuckerberg is relying on the cash generated by the company's dominant ad business to fund his massive bet on artificial intelligence , which could cost as much as $145 billion this year.
The judge in New Mexico ordered Meta to pay $567 million into an abatement fund as part of a second phase of a case involving child sexual exploitation allegations. A jury there ruled in March, during the trial's first phase, that Meta must pay $375 million for violating the state's unfair practices act. Meta said it disagreed with the New Mexico ruling and plans to appeal.
Torrez called it a "pretty substantial judgment," but was quick to note how it pales in comparison to what could be coming.
"This is a state with some 2 million people," Torrez said. "If you map that same argument onto California or Florida or Texas or New York, I mean, that's a potentially massive and a market shifting force."
Many of the specifics of the cases vary, but they all center on the alleged harmful design of apps like Facebook and Instagram.
In New Mexico, Meta is being required to improve "age assurance models and tools" using AI, and to attempt "to develop, within two years, a dedicated under-13-years-of-age prediction model." Other remedies include making it easier to report underage usage, and partnering "with schools or a child safety organization to create a reporting portal where school administrators can flag suspected accounts where users may be under 13 years of age across any social media platform."
Torrez said that plaintiffs' focus on app design features and alleged misrepresentations on safety "really provides a blueprint for other states to hold them accountable." It's an approach that allows states to avoid Section 230 of the Communications Decency Act , which has generally shielded tech companies from legal exposure due to third-party content on their platforms.
Meta and Google's YouTube lost a case in March, when a jury in Los Angeles determined the companies were negligent and failed to warn users of the dangers associated with using their platforms.
California Attorney General Rob Bonta speaks at a press conference in February 2024.
CNBC
Bonta, who was appointed to the AG role in 2021 by Gov. Gavin Newsom, said in a statement last Monday that one of his most important jobs is to "protect our children from harm."
"Meta designed a dangerous product for young users, knew it to be dangerous, and then lied to children, families, and the community about how dangerous it was," he said. "We are ready to hold Meta accountable for its role in fueling the mental health crisis of American children and look forward to trial."
Meta said in a statement that the AG coalition's "limited claims are unsubstantiated and their financial demands are vastly disproportionate."
"The AGs offer no proof anyone in their states was misled, claim benign features like having an additional Instagram account somehow harmed their residents, and attempt to penalize Meta for industry-wide challenges like age verification," Meta said.
Meta's attorneys have previously said that the consolidated state AG trial could lead to damages as high as $1.4 trillion. Lawyers representing the states told presiding federal Judge Yvonne Gonzalez Rogers last week that $200 billion is a more likely amount.
"You could wake up with a headline judgment that is, as I've said, astronomical," Torrez said.
Michael Coffey, a defense litigator and founding partner of New York law firm Coffey Modica, said it's the kind of case that can define the career of a state AG, while the victims could potentially end up getting very little of the ultimate payout.
"It's all going to go to government, and then they're going to dole it out, and then the plaintiffs' bar is going to take a cut out of it," Coffey said.
Money aside, the potential design changes Meta would have to implement if the company loses in California could be far more damaging, said Laura Marquez-Garrett, an attorney with the Social Media Victims Law Center.
"They have the ability that private plaintiffs typically do not have to actually force these companies, through the court system, to change their business model, their design decisions, all of that," Marquez-Garrett, who worked on the recent Los Angeles case , said during a Friday media briefing.
The state AGs said in a court filing that they're seeking "permanent injunctive relief, on a nationwide — rather than state-by-state — basis, to restrain Meta's unlawful acts and practices" for any violations related to COPPA. If Meta is found to have violated the federal law, the states want the company to delete all of the personal data for children under 13 as well as the "algorithms and models" that were trained using that information.
Regarding any violations of state consumer protection laws, the attorneys representing the states said in the filing they want Meta to be forced to remove "certain addictive design features," including infinite scroll, autoplay, ephemeral content, beauty filters and "engagement-optimized algorithms."
Torrez conceded that his team didn't get all the design changes it was seeking, like eliminating infinite scroll and altering the company's recommendation algorithms. The judge in the case said that some of those changes could conflict with Section 230 and First Amendment protections , and that imposing them would be unfair because rivals like TikTok and YouTube would still retain the features.
Torrez said he plans to turn to the legislative process with a social media bill and consumer protection law updates that are intended "to capture a whole range of business practices that are common in the digital economy."
Meta and its social media rivals face multiple other lawsuits around the nation, including a big federal trial expected to commence in Northern California next year involving a collection of school districts. The first of multiple consolidated school district trials was set to begin this summer, but Meta, YouTube, TikTok and Snap all settled .
The way Torrez sees it, Wall Street is underestimating the potential significance of a loss in California and is just looking at New Mexico "in isolation." Meta's stock is down 11% this year, but most of the concern expressed by analysts has to do with the company's massive capital expenditures bill for AI infrastructure rather than fears about a potential deterioration in the ad business.
"The analysts aren't pricing this correctly right now," Torrez said. "That California judgment by itself could be gargantuan enough that it changes the ability of this company to do what it needs to finance into the future."
WATCH : Meta's federal trial moves forward, accused of violating children's privacy laws .
Workers cut behind the slow-moving tractor and packing platform. These workers use shorter knives, rather than long awkward machetes, bend into those big wet leaves, trimming and tossing the heads onto the conveyer belt. | David Bacon
In the winter of 1977, at the Watsonville union hall, I got a dispatch to a job in a Valley Harvest cauliflower crew. Two years earlier the state legislature had passed the Agricultural Labor Relations Act. The company was one of the first where workers used the new law, winning a union election, a contract, and with it the hiring hall system for distributing work.
That year I left my job as a UFW organizer and went to work in the fields. I needed the money, and wanted to understand firsthand agriculture's often brutal labor. I worked through that fall, getting a dispatch first for a painful job picking strawberries, and then another to harvest wine grapes on the piece rate at the huge Almaden Vineyards. Working on the piece rate you're paid according to what you produce, in this case, the quantity of grapes we picked.
When the grapes were gone, I needed another job. Cutting cauliflower for Valley Harvest was one of the few available in December. It paid by the hour, not much more than minimum wage. That meant that I wouldn't make as much money as I did at Almaden, but working for an hourly wage avoided the intense pressure of the piece rate. I was never a very fast worker anyway.
So every morning I'd leave my SRO hotel room in Hollister, pick up my lunch burritos and a cup of coffee from the bar downstairs, and drive the 23 miles to the packing house parking lot in Pajaro. With the other 40-plus workers in my crew, I'd get on the company bus for the ride to the field. We'd sleepily stumble out into the cold fog, put on our aprons, and use one of the company stones to sharpen up our machetes for the day's work.
As the mist burned off the coastal fields, we'd walk behind a conveyor belt and packing platform, pulled by a tractor. The giant leaves of the plants were covered with dew, and bound together with long rubber bands. This practice, called blanching, protected the cauliflower heads from the sun, making sure they stayed the white color favored by consumers at Safeway. The job was to grab the leaves, cut the stalk with a machete, and toss it onto the belt. Other workers there would trim the leaves, wrap the heads in plastic, and pack them into boxes.
One morning we looked for the stones, and they weren't there. The foreman announced that in a fit of economizing, Valley Harvest had decided not to furnish them. If we wanted sharp machetes, we'd have to fend for ourselves. The stones couldn't have cost the company much, but they were crucial. Trying to cut a tough, thick cauliflower stalk with a dull blade makes the work twice as hard. We were pretty angry.
We worked through that day, and at lunch some of us talked it over. I said, "Well, we need to do something, so maybe we should tell the company we're not going to work unless they give us the whetstones back." Someone else said, "Yeah, but what about those folks over there, the Oaxacans?" They were a close-knit group that would eat a little ways away from the rest of us, talking softly with each other in an unfamiliar language. Later I learned they were speaking Mixteco, an indigenous language of southern Mexico.
I went over to talk to them, and explained in Spanish what we wanted to do. I asked, "What about you folks? Are you going to be with us or not?" One of them answered, "Well, let us talk about it and we'll get back to you." Then they went off by themselves and had a little meeting in the road by the field. When they came back he said, "Yeah, we're with you. Let's do it."
The following morning we got out of the bus and refused to go into the rows. The foreman suddenly discovered the box of stones, still under a seat. So we got them back, sharpened our knives, and went to work. That day I learned a little about the indigenous people arriving from Oaxaca, especially that they were already very well organized. They knew how to act collectively, thanks to the indigenous culture of the towns they came from.
Today several hundred thousand Indigenous farmworkers from southern Mexico labor in fields, not just in California, but all over the U.S. Their culture, and their political and labor activism, are changing rural communities profoundly. They've educated me for fifty years, an education that started in that field.
Not long ago I came upon another cauliflower crew working on a Salinas Valley farm. It took a while, and several phone calls, to get permission from the foreman, but finally I was able to start taking pictures. I joked with the workers, asking their permission also, as I tried for photographs that would show the intimate details of their work.
I saw that in 50 years it hadn't changed that much. Workers still cut behind the slow-moving tractor and packing platform. These workers used shorter knives, rather than our long awkward machetes, and trimmed the leaves from the cauliflowers before putting them on the conveyor belt. But bending into those big wet leaves, cutting and tossing the heads onto the belt, was the same. I remembered also how my back would feel on fire, after hours of this labor, and I'm sure that was the same as well.
The biggest change is that growers have bred a variety of cauliflower that produces leaves that bend inward to shade the head. Now they get that white color, and don't have to pay workers to put the blanching bands around the plant. That job, which used to help families survive the slow season, no longer exists.
The idea that a sun-tanned cauliflower head is unattractive to consumers actually leads to waste on many farms. Heads exposed to the sun taste the same, and may actually contain more phytonutrients - chemicals that help plants defend themselves against insects and disease, and that benefit humans as well. But Richard Melvin, a Nova Scotia cauliflower grower, estimates that up to 40 percent of his crop is plowed back into the ground because of its "off" color.
Nevertheless, the crew in the Soledad field, cutting for the Nature's Reward brand of the Huntington family, didn't seem to leave many heads behind. A lot of growers also grow orange, purple and green varieties. The chemicals that create the colors include beta-carotene, chlorophyll and anthocyanins, which are antioxidants and healthy for humans. But even white cauliflower provides Vitamin C and folate, sun-tanned or not.
I used to steam my cauliflower whole and smother it in cheese sauce until my wife became lactose-intolerant. Now we usually cut the head into florets, and roast them with a sprinkling of breadcrumbs and parmesan cheese, dusting them with a little chile spice mix. When I eat them I think of the fields where I worked near Watsonville, and appreciate the labor of the men and women who cut the cauliflower today.
David Bacon @photos4justice on the daily lives and ongoing struggles (both personal and political) of farmworkers - interview on Against the Grain with C.S. Soong
We have released
dotenv-ng
1.0, a
modern Rust implementation for loading and rendering
.env
files. It began as
a fork of
dotenvy
after its parser
changed a secret while reading it.
That may sound contradictory. SecretSpec is still on a mission to
eliminate
environment variables as a secrets
interface
, and we have written about
where
.env
went wrong
. It should not be the
final home of a secret.
But migrating away from
.env
starts with reading it
correctly.
The immediate failure was SecretSpec issue #73 . A dotenv file contained a value with bcrypt fragments:
TEST="foo:$2a$10$TWoviNHS27HJMw1PKe4tBeIMlms6tWdYS9hKoHANKCQhluDlEt/gu"
The file was intact. Reading it through the dotenv provider returned a
different value because
dotenvy
treated the dollar-prefixed fragments as
variable substitutions. The failure appeared later as an authentication error,
not a parse error.
An upstream request to make substitution configurable had been open since 2024 . A pull request arrived in 2026 but targeted an unreleased API. A migration tool cannot require users to recognize and escape parser syntax inside their secrets.
The original Rust
dotenv
crate stopped releasing in 2020 and was eventually
marked
unmaintained by
RustSec
, which listed
dotenvy as an alternative.
Dotenvy’s description still calls it “a well-maintained fork.” Its latest published version, 0.15.7, was released on March 22, 2023 . A Rust forum discussion noted the two-year release gap in 2025. By the time the bcrypt bug blocked SecretSpec, it was more than three years.
There is an uncomfortable irony in a maintained fork repeating its upstream’s release problem. Its maintainers do not owe us a release, but SecretSpec needed breaking fixes on a schedule we control.
We first considered a small patch. Auditing the parser uncovered more problems around JSON, Windows paths, Unicode names, precedence, and partial environment mutation.
dotenv-ng
therefore starts from dotenvy 0.15.7 but deliberately breaks
compatibility where correctness requires it. Version 1.0 adds:
unsafe
boundary around that mutation.
Property tests exercise arbitrary Unicode and syntax-heavy values, check that quoting is used only when necessary, and round-trip complete documents. The parser and renderer, the core of the rewrite, both have 100% line coverage.
The complete compatibility and API changes are recorded in the
dotenv-ng
1.0 changelog
.
The package is available on
crates.io
.
Applications can keep the familiar
dotenv
crate name with a dependency
alias:
[dependencies]
dotenv = { package = "dotenv-ng", version = "1" }
Starting in SecretSpec 0.20, dotenv-ng powers dotenv parsing and rendering throughout SecretSpec.
Abstract: High-performance GPU programming has traditionally forced a compromise between execution efficiency and memory safety. While Rust guarantees compile-time memory safety for host CPUs via its strict ownership model, applying these constraints to massively parallel GPU execution environments has previously mandated either vendor-locked Domain-Specific Languages (DSLs) or escaping to explicit unsafe raw pointers. This paper presents a zero-overhead, multi-vendor GPU compilation framework built natively into the Rust compiler (rustc) and LLVM backends.
We leverage Rust's rich type system, ownership system, and strict aliasing guarantees (noalias) to efficiently manage and optimize data transfers through LLVM's Offload infrastructure. We expose the technical challenges of cross-vendor ABI lowering mismatches between Host and Device targets and introduce a two-pass compilation pipeline capable of safely handling both manual and compiler-generated memory movements. Evaluating our framework on RAJAPerf demonstrates that our rustc-based solution can generate competitive LLVM IR for GPU kernels, achieving a solid kernel performance against native, hand-optimized CUDA and HIP C++ baselines.
From: Manuel Sebastian Drehwald [
view email
]
[v1]
Thu, 13 Aug 2026 20:37:48 UTC (88 KB)
Artificial Analysis Intelligence Index v4.1.1 incorporates 9 evaluations: GDPval-AA v2, 𝜏³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GPQA Diamond, CritPt, AA-Omniscience, AA-LCR
Reasoning models are indicated by a lightbulb icon
Artificial Analysis Intelligence Index v4.1.1 includes: GDPval-AA v2, 𝜏³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GPQA Diamond, CritPt, AA-Omniscience, AA-LCR . See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Artificial Analysis Intelligence Index v4.1.1 incorporates 9 evaluations: GDPval-AA v2, 𝜏³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GPQA Diamond, CritPt, AA-Omniscience, AA-LCR
Reasoning models are indicated by a lightbulb icon
Artificial Analysis Intelligence Index v4.1.1 includes: GDPval-AA v2, 𝜏³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GPQA Diamond, CritPt, AA-Omniscience, AA-LCR . See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Indicates whether the model weights are available. Models are labelled as 'Commercial Use Restricted' if the weights are available but commercial use is limited (typically requires obtaining a paid license).
Intelligence evaluations measured independently by Artificial Analysis · Higher is better
Quantitative analysis on spreadsheets & documents
Reasoning models are indicated by a lightbulb icon
While model intelligence generally translates across use cases, specific evaluations may be more relevant for certain use cases.
Artificial Analysis Intelligence Index v4.1.1 includes: GDPval-AA v2, 𝜏³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GPQA Diamond, CritPt, AA-Omniscience, AA-LCR . See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
AA-Omniscience Index (higher is better) measures knowledge reliability and hallucination. It rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer. Scores range from -100 to 100, where 0 means as many correct as incorrect answers, and negative scores mean more incorrect than correct.
Reasoning models are indicated by a lightbulb icon
AA-Omniscience Index (higher is better) measures knowledge reliability and hallucination. It rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer. Scores range from -100 to 100, where 0 means as many correct as incorrect answers, and negative scores mean more incorrect than correct.
Openness Index assesses model openness on a 0 to 100 normalized scale (higher is more open)
Reasoning models are indicated by a lightbulb icon
Artificial Analysis Intelligence Index · Weighted average cost (USD) per Artificial Analysis Intelligence Index task
Reasoning models are indicated by a lightbulb icon
Weighted average cost per Intelligence Index task. Each evaluation’s cost is calculated from input, cache hit, cache write, reasoning, and answer token prices, divided by task count, and weighted by its Intelligence Index weight.
Artificial Analysis Intelligence Index v4.1.1 includes: GDPval-AA v2, 𝜏³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GPQA Diamond, CritPt, AA-Omniscience, AA-LCR . See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Weighted average number of output tokens used to run one task in the Artificial Analysis Intelligence Index
Reasoning models are indicated by a lightbulb icon
The number of tokens required per Intelligence Index task. This is calculated by multiplying the output tokens per eval by the relative weights of each benchmark in the Intelligence Index, then dividing by task count (excluding repeats).
Weighted average cost (USD) per Artificial Analysis Intelligence Index task, segmented by token type. Lower is better
Reasoning models are indicated by a lightbulb icon
Weighted average cost per Intelligence Index task. Each evaluation’s cost is calculated from input, cache hit, cache write, reasoning, and answer token prices, divided by task count, and weighted by its Intelligence Index weight.
Cost (USD) to run all evaluations in the Artificial Analysis Intelligence Index
Reasoning models are indicated by a lightbulb icon
The cost to run the evaluations in the Artificial Analysis Intelligence Index, calculated using the model's input, cache hit, cache write, reasoning, and answer token prices and the number of tokens used across evaluations (excluding repeats).
Price (USD per M Tokens)
Reasoning models are indicated by a lightbulb icon
Price per token for cached prompts (previously processed), typically offering a significant discount compared to regular input price, represented as USD per million tokens. The values shown here are the cache hit price; cache write and cache storage are billed separately and vary by provider — see "Cache pricing by provider" for detail.
Context window: tokens limit · Higher is better
Reasoning models are indicated by a lightbulb icon
Larger context windows are relevant to RAG (Retrieval Augmented Generation) LLM workflows which typically involve reasoning and information retrieval of large amounts of data.
Maximum number of combined input & output tokens. Output tokens commonly have a significantly lower limit (varied by model).
Comparison between total model parameters and parameters active during inference
Reasoning models are indicated by a lightbulb icon
The total number of trainable weights and biases in the model, expressed in billions. These parameters are learned during training and determine the model's ability to process and generate responses.
The number of parameters actually executed during each inference forward pass, expressed in billions. For Mixture of Experts (MoE) models, a routing mechanism selects a subset of experts per token, resulting in fewer active than total parameters. Dense models use all parameters, so active equals total.
It’s been a few months since my post on leaving Gmail which sparked a lot of discussion on Hacker News . Email isn’t the most exciting topic in the world, but I figured I should give a little update on how things have been going after the move to Fastmail since so many people saw the original post.
The quick version is that I think I picked right with Fastmail. You can find cheaper email hosts out there. I’m sure they’re great too, but I’m really happy with Fastmail’s value considering all I’m getting.
I ended up starting fresh instead of forwarding my Gmail to my new address. This was definitely the right move for me for reasons I’ll get into.
My biggest concern moving away from Gmail was my inbox organization. Gmail does a pretty alright job of automatically sorting out promotional emails from notification emails from human-sent emails and so on. It’s the only reason my inbox has remained somewhat manageable over the years despite no effort on my part.
The solution for this that I landed on is subdomain addressing and so far it’s been keeping me even better organized than before. I’ve updated my important accounts with unique or category-based subdomain addresses, then any mail sent to those addresses automatically gets moved to a matching folder without me even having to create any rules.
If I had forwarded my Gmail to the new Fastmail address, organization would have been a whack-a-mole rule creation game.
Going and updating my accounts with a new address was easier than I was expecting. It does help that I’m keeping the Gmail address active as a junk inbox that I still occasionally check though, so I’ve only had to update the accounts that I actually care about. It wasn’t more than ten or so accounts, and I’m getting to the rest gradually as I remember about them.
Fastmail lets you connect something crazy like 100 domains to your account. That was a nice bonus for me because I was able to add a new address using my blog domain, something I’d been wanting to do for a while anyway.
This is probably a coincidence, but I’ve been getting way more email replies to my posts since I updated my contact page with my new address on my blog domain. I used to have a Proton address I created specifically for email replies listed there. It’s been a fun new motivator for blogging, and it’s also helped me remember to reach out when I read a cool post.
Since I’m talking about custom domains already, here’s something you might want to know if you’re thinking of making a move similar to mine. You’ll probably find your sent emails “greylisted” by your recipients’ email providers if your domain has been freshly registered — regardless of your email host. For example, emails sent from my brand new domain were getting delayed by a few hours when sent to Gmail addresses. Messages from my blog’s domain, which has been around for a while, were getting delivered immediately. It took a day or two for Gmail to fully trust the new domain and remove the delay.
This is something I’m using more than I thought I would. You can generate randomized addresses that route to your inbox so you don’t have to share your real address. It’s nice to use for services that you may want to easily block, since you can flip a toggle and start sending all mail received at a masked address to the trash.
There’s a bunch of other little things I’m liking about Fastmail too. Their apps are solid. Their documentation is good. I haven’t really found anything not to like yet.
The main thing I want you to take away is that the switch has been much smoother than I expected. If you’re thinking of making a switch too — to anywhere, not just Fastmail — I don’t think you have a whole lot to worry about. Especially if you move to an address on your own domain, then any other moves in the future will be even easier.
— JP
Codeberg , a Git code hosting platform, recently took a decision to prohibit projects that mostly consist of generative-AI-written code which has prompted concern and extensive discussion elsewhere .
The decision does not surprise me, and I don’t mean that as a criticism. Codeberg has always presented itself as a mission-driven alternative to GitHub, not neutral infrastructure. 1
What interests me is the disappointment in the response. Many people reacted as though one of the few plausible GitHub replacements had ruled them or their projects out. They wanted Codeberg to be a universal alternative, a better GitHub and the obvious place to go when leaving it.
To me, that exposes a big gap in the open-source space. There are plenty of places to host a Git repository, but remarkably few places to host an open-source community. GitHub gives projects a shared pool of identities, habits and paths to discovery. None of the alternatives has reproduced that at a similar scale.
I don’t think the answer has to be another centralized platform, or that every project should live in one place. But decentralization is not enough on its own. Whatever replaces GitHub still needs a shared social layer: identities contributors already have, conventions they understand and ways to discover projects across the network.
Whenever dissatisfaction with GitHub comes up, someone inevitably says: “Git is decentralized. Just self-host a forge.”
I’ve self-hosted Gitea for years, so this is an argument I’m very familiar with. Self-hosting works well for personal projects, but I wouldn’t use it for something I wanted strangers to contribute to. On GitHub, most people already have an account and understand how issues and pull requests work. On my forge, even reporting a small bug means creating another account, learning how my forge works and what conventions I want you to follow. Unless someone really cared, they probably wouldn’t bother. I know I wouldn’t.
And contribution is only half of it. GitHub used to be genuinely good at discovery. I regularly found projects because someone I followed starred them, often in areas I would never have searched for myself. It felt like a social network built around people making things.
GitHub has since redesigned that feed, and I almost never visit it anymore. Defaults are powerful: once discovery stopped being part of the experience GitHub put in front of me, it largely disappeared from my workflow.
The basic experience of GitHub has been getting steadily worse. It is slow, things regularly fail to load and notifications are unreliable. GitHub itself recently described two major incidents as “not acceptable” .
Its pull request experience has been awful too. Large PRs are painfully slow to navigate and review. Stacked PRs 2 have been common inside large software companies for well over a decade, but only just became a thing with GitHub and, even then, seems to be quite buggy .
What frustrates me about GitHub’s push towards AI is that the core forge feels neglected while Copilot appears everywhere. An agent writing more code doesn’t help when the interface for reviewing it is already struggling.
Ghostty exemplifies this frustration. In late April, Mitchell Hashimoto announced that Ghostty is leaving GitHub because frequent outages were preventing its maintainers from working reliably:
On the day I am writing this post, I’ve been unable to do any PR review for ~2 hours because there is a GitHub Actions outage. This is no longer a place for serious work if it just blocks you out for hours per day, every day.
Interestingly, he also makes the point that GitHub is more than hosting:
To the “Git is distributed!” crowd: the issue isn’t Git, it’s the infrastructure we rely on around it: issues, PRs, Actions, etc.
Hashimoto said Ghostty was in discussions with multiple commercial and FOSS providers and planned an incremental migration. The fact that such a prominent project had to shop around, rather than move to an obvious default, is exactly the gap I mean.
It’s worth going through the alternatives and the problems I see with each:
GitLab is capable, but it feels incredibly corporate, even more so than GitHub. 3 Nor have I found it as good as GitHub at helping people stumble across projects and developers.
SourceHut is focused and transparent and, like Codeberg, openly values-driven. 4 Its email-oriented workflow, while battle-tested by projects like the Linux kernel, is unfamiliar to most GitHub users.
Forgejo ’s federation project may eventually connect self-hosted instances into a shared network. It looks promising, but has been in development for quite some time, remains experimental 5 and is not yet a practical answer to the social fragmentation of self-hosting.
Radicle is technologically interesting: repositories are replicated peer to peer, while issues and patches are stored alongside them. But it still feels too immature to replace GitHub for a public project. For example, its public web interface lets people browse repositories, but contributing requires them to install its CLI or desktop application. Someone encountering a bug should not have to install the forge’s software merely to report it. 6
The project I’m most interested in is Tangled , based mostly on gut feel. I like its focus on the social experience around code. For example, its home page immediately shows me a bunch of cool projects, exactly like old GitHub used to do. But it is still in alpha, and it remains to be seen whether it can blossom into a true alternative.
One thing that stands out is that, apart from GitLab, every competitor is built around either decentralization or a social mission. That makes each of them fundamentally less “straightforward” than a for-profit company. It makes me wonder whether there is room for one.
Perhaps it could look more like bunny.net than a venture-backed startup: a deliberately boring company with no ambition to become the operating system for software development or to reorganize programming around whatever technology investors currently find exciting. It might simply concentrate on making open-source collaboration pleasant and reliable, charge developers and smaller organizations directly, and grow at whatever pace that revenue supports.
The open question is whether this could be a viable business. The difficult part is exactly what I keep saying is missing: shared identity, shared conventions and discovery only become valuable once a platform has reached scale. Better repository hosting alone would not solve that cold-start problem. 7
I don’t know who, if anyone, will solve it. But a real replacement will have to treat the social layer as the product, not as something that appears automatically once enough repositories are hosted.
The world is familiar by now with the usual tropes of machine-generated text: overuse of the word “delve”, an excess of em dashes, and the chirpy, relentless construction of “it’s not X but Y” .
But is it about to get even worse?
Over the weekend, Anthropic released an update saying it would change how its Claude AI model generated prose. This is to comply with an EU regulation that requires all AI-generated text to be watermarked starting in December.
Anthropic said the changes would be made at the granular, random level at which its models generated text and would be undetectable to the average reader.
But at least one commentator thinks otherwise. “This entire endeavour is a perverse adulteration of what it means to write,” wrote John Gruber , a veteran tech blogger. He argued the watermark would constrain Claude, forcing it to make worse, less precise word choices overall. While it may not make the model’s writing less accurate, he suggested these limits would make it worse.
There is a stochastic element in the small choices that an AI model makes in framing a sentence: whether it chooses to call a day “grey” or “overcast” , for example, or refers to a running water as a “stream” or a “brook”.
Anthropic’s new watermark will alter these random choices, it said, leaving a pattern that will be detectable to Anthropic itself, and to those who have a key to decode it.
Steven Murdoch, a professor of computer science at University College London, said the change “probably wouldn’t have any noticeable impact”.
Gruber’s complaint appears to centre on the fact that watermarking will make a large language model less free to decide which word comes next in a sentence, and may therefore not pick the best choice.
However, LLMs already do not make the best choices. “There’s already randomness involved in any of these large language models,” said Murdoch. “It’s pretty essential to how they work. If it wasn’t for this randomness, then they’d get stuck in loops and start repeating the same thing over and over again.”
In other words, a chatbot does not necessarily decide to call running water a “stream” as opposed to a “brook” because the latter might have a more old-fashioned register. The model does not contemplate these choices: it makes them by chance.
This was evidenced recently when a paper in a leading chemistry journal had to be retracted because the authors appeared to have used an AI tool to draft part of the publication. The tool used the phrase “mass killing of an ethnic group” as an alternative for “final solution”, though the paragraph in question appeared to be about a zinc nanogel.
On the update, Murdoch said: “There’s going to be no noticeable difference. There’s the same random number generators there – it just used to be completely random, and now it’s statistically predictable, but still random.”
The regulation, which applies to all AI companies operating in the EU , means they will have to put in place these watermarks within months. This could make it harder for students, lawyers and university professors to pass off chatbot-written content as their own .
There is another reason to watermark AI content, though, Murdoch said: there’s so much of it already out there that it might damage the models themselves. Training AI on AI-written content creates “model collapse”, leading models to confuse concepts.
In other words, watermarking is not just a quietly powerful way to combat disinformation – it’s a deeply transformative system to ensure the chatbots don’t go insane. If you delve into it.
A watermark in plain text sounds impossible. Text has no pixels to hide data in, and no metadata survives copy-and-paste; every character is right there in front of you. Where could a mark possibly go?
And yet the marks are real. Google has watermarked text from the Gemini app and web experience since 2024 (its API is, at the time of writing, a documented exception ), and as of August 2026, new Claude models mark text at the model level, with earlier models to follow. They're invisible, they survive copying, and they work because they don't live in the characters at all. They live in the choices between words .
The one idea in this step: a model writes by rolling weighted dice between several words that would each be fine.
When a model is mid-sentence, it doesn't know "the next word." It has a shortlist, like autocomplete, with preferences. Here's a real kind of moment, one word from the end of a sentence:
the sentence being written
The results of the study were quite
Each roll sweeps the shortlist, lands on one word (odds matching the bars) and drops it into the sentence above. The dots tally where the rolls land: try ×20 and watch the pile take the shape of the odds. Notice what never changes: every landing makes a perfectly good sentence.
A page of text contains hundreds of these little forks, one per word, and at many of them several options are equally fine. That slack is the raw material. Whoever gets to lean on how the dice land can hide a pattern in the text without changing what it says.
The one idea in this step: the key secretly colours the shortlist and gives one colour a gentle nudge. The text still reads normally.
Here is the classic recipe (Kirchenbauer et al. 2023; Google's SynthID reaches the same end by a subtler, tournament-style route). At each fork, secret-keyed maths splits the candidate words into green and red , an arbitrary colouring only the key-holder can reproduce. Then the dice get tilted a little toward green.
the sentence being written
The results of the study were quite
No key applied: these are the model's own preferences. Dashed outlines will show the old odds once the key is on.
Two things make this sneaky. The nudge is mild: a red word can still win — it's just a little less likely. And the colouring is not a fixed property of the word: the key computes it from a short run of the words just before, so the same candidate is green after one prefix and red after another:
The same four candidate words, coloured by the key after six different prefixes. The key sees the words just before it; the position in the wider text is invisible to it. Only the overall lean toward green accumulates, and only the key-holder knows which words were green where.
(Two siblings, same principle. Google's SynthID — the one in production — replaces the nudge with a tiny secret tournament : a few candidates are drawn from the model's own odds, the key scores them, and the bracket is arranged so that, averaged over the key's draws, every word's odds stay exactly what the model intended. Aaronson's scheme, built at OpenAI, skips even that and derives the dice-rolls themselves from the key. Different maths, same principle: the mark lives in the choices.)
The one idea in this step: with the key, you can re-colour any text and simply count. Marked text lands green too often to be luck.
Detection doesn't read the text or judge its style. The detector replays the key-holder's colouring over the words and counts how many came up green. Without a mark (or without the right key), green should win about half the time. A coin flip. Here's an ordinary-looking paragraph; try both keys on it:
greens: – of 55
coin flip
flag bar (this length)
Filled-and-underlined chips are green, dashed outlines are red. The words read identically either way; the colouring exists only in the key-holder's maths. With the wrong key the split is meaningless, and the count sits at chance.
(This demo's tilt is drawn strong so you can see it; a production mark leans far more gently and needs correspondingly more text. In this demo's 50/50 model, a 1,500-word document would flag at only ~55% green: small leans become persuasive only through length, which is why short texts are genuinely hard to call.)
The one idea in this step: the mark lives in runs of untouched wording. Editing erases it exactly where the runs break, and nowhere else.
Each word's colouring is derived from a short run of the words just before it (one to a handful, depending on the scheme). So a position only counts as evidence if a short window of the original wording (the word plus its neighbours) survives intact.
Here is the same paragraph from step 3, at five edit depths. Drag the slider and watch the highlighted runs shrink. A highlight means that run of wording still matches the original exactly, so the detector can count there. Everything faded is new wording, where there is nothing but coin-flip noise left to count.
fix typos · surviving windows: – %
The verdict reads the surviving fraction measured from the highlights above , projected to a 1,500-word document. Two things to notice: how much a "heavy edit" leaves standing, and how far toward a full rewrite you have to drag before the evidence actually dies.
On real implementations (MarkLLM's KGW and EXP schemes on an open model, washed by declaude's full-rewrite route): about 0.5% of windows survive, and detector accuracy falls from essentially certain to a coin flip. The published literature agrees on the shape of this. Light or one-pass paraphrase dilutes the mark rather than deleting it; in Kirchenbauer et al.'s experiments, the detector recovers given enough text, with even human paraphrase becoming detectable again after roughly 800 tokens (about 600 words). What removes the mark is re-composition that shares no runs of wording with the original.
That is why a tool that rewrites from the meaning (like declaude 's full-rewrite route) is what actually erases this family of marks, and why a light pass that keeps most of the phrasing does not.
One boundary stated plainly: those numbers come from open implementations we can measure. Anthropic's production scheme is undisclosed, so no one outside Anthropic can yet run this test against Claude's own mark.
The one idea in this step: detection is private, probabilistic, and about processing , not authorship.
Written by James Padolsey at NOPE as an accompaniment to declaude . The interactive figures are a teaching model with illustrative parameters, not any provider's actual scheme.
Sources & further reading. Kirchenbauer et al., A Watermark for Large Language Models (ICML 2023) · Dathathri et al., Scalable watermarking for identifying LLM outputs (SynthID-Text, Nature 2024) · Aaronson & Kirchner, Watermarking GPT outputs (2022) · Kirchenbauer et al., On the Reliability of Watermarks for Large Language Models (ICLR 2024) · Sadasivan et al., Can AI-Generated Text be Reliably Detected? (2023) · Zhao et al., The Mark Fades: Adaptive Evolutionary Paraphrase-based Attack (ACL Findings 2026) · Anthropic, How Claude marks AI-generated content (Help Center, Aug 2026) · Our own known-key experiments: re-composition collapses KGW/EXP detection to chance (AUC 0.99 → ≈0.5), context-free unigram marks survive (0.73–0.84); outline-level regeneration is the only answer we know for meaning-space marks.
For the specialist: the residual-evidence model behind the step-4 verdict is z ≈ f·√N·z₁ (surviving fraction f, document length N, per-token strength z₁). The figures count words; real detectors count tokens in the model's own tokenizer. Same shape.
Your next group birthday gifts should be home batteries.
Most birthday gifts are forgotten within a year. Batteries save your friends money every day and keep essential devices running when the power goes out.
A 5 kWh battery costs about €1,600. That is expensive for one person and easy for a group:
Create a battery birthday club. Buy each friend a battery on their birthday until everyone has one.
The idea is simple: buy electricity when it is cheap, store it, and use it when it is expensive. Automatic price optimization reads tomorrow’s prices and handles the schedule for you.
The EcoFlow STREAM 5000 is one current example: about €1,600 for 5 kWh of storage and up to 3 kW of output.
Germany and Spain are good places to do this. In the second half of 2025, the average household electricity price was €0.3869/kWh in Germany and €0.2669/kWh in Spain, including taxes. Those averages hide large changes during each day.
I calculated what would have happened on every day of 2025. The battery buys 5 kWh during the four cheapest hours before 17:00 and delivers 4.5 kWh during the four most expensive evening hours. That assumes 90% round-trip efficiency and only runs the battery when the cycle is profitable.
| Country | Average cheap price | Average expensive price | Average saving per day | Saving in 2025 | €1,600 payback |
|---|---|---|---|---|---|
| Germany | €0.041/kWh | €0.133/kWh | €0.40 | €144 | 11.1 years |
| Spain | €0.026/kWh | €0.106/kWh | €0.34 | €126 | 12.7 years |
The calculation is:
daily saving = 4.5 × expensive price − 5 × cheap price
The table uses the market-linked part of a dynamic tariff, based on German SMARD and Spanish OMIE day-ahead prices. Your bill also contains supplier charges, network fees, and taxes. Some are fixed and some depend on time and location, so you should run the same calculation against your actual tariff. The national household price comparison comes from Eurostat .
Grid arbitrage alone is therefore useful, but not yet an automatic five-year payback. The economics improve when the same battery also stores your solar power, earns grid-service payments, or replaces electricity bought at the full retail price.
You are also safer during a power outage. The STREAM 5000 provides up to 3 kW of off-grid output. Keep essential devices on its backup output and reserve some battery capacity, and your refrigerator, lights, router, and laptop can keep running when the grid goes down. An ordinary grid connection does not back up the whole house automatically; the backup output must be configured correctly.
And when the neighborhood goes dark but their lights stay on, they will remember who gave them the battery.
A battery also decouples when electricity is produced from when it is used. Spain can save abundant midday solar for the evening. Germany can save wind power produced during low demand for the next peak. More low-carbon energy can be used instead of wasted simply because it arrived at the wrong hour.
Ten million 5 kWh batteries would create 50 GWh of distributed storage and 30 GW of output. At one cycle per day, they could move about 16 TWh of electricity each year.
Sun Clock is a 24-hour clock that displays the position of the sun, and times of sunrise , solar noon , sunset , golden hour , and twilight for your current location. It also shows the position and phase of the moon, and its rising and setting times.
A note about direction 1 — why does it go backwards?
Tap on or hover over the segments to get their start and end times. You can also tap/hover on the moon, the hour hand, and the centre dot.
See updates for change history.
Sun Clock is free to use, and contains no advertising. If you would like to help support Sun Clock, please —
We collect aggregate user stats only. Your location and settings are stored in your web browser and are not sent to the server. No cookies are saved or sent.
In the Northern Hemisphere the Sun moves across the sky in a clockwise direction. (Before clocks, clockwise was called " sunwise ", and anti-clockwise was know as " widdershins ", meaning "against the way".)
In the Southern Hemisphere, however, the sun moves across the sky in an anti-clockwise direction. Sun Clock matches this by setting its direction of rotation based on your latitude: if you're in the Southern Hemisphere the clock will go 'backwards' . You can change it in the settings if you wish.
Ideally you want the clock to turn in the same direction as the sun, regardless of which hemisphere you are in. If you are facing South, set it to clockwise; if facing North, anti-clockwise. You want sunrise on the clock to be to the East.
In fact, if you face the right direction and tilt you screen at the right angle, the hour hand will track the movement of the Sun across the sky. Exactly how to do this is left as an exercise for the reader, but you want the screen to lie in the plane of the ecliptic , and solar noon to be "up".
Added a license (MIT) to the code.
Added an option for a ticking seconds hand (sweep hand is still the default).
Fixed a bug where the Moon icon was incorrect in recent versions of Safari.
Added the option to show the odd numbers on the clock face.
The " use 12-hour times " option now applies to the numbers on the clock face also.
Added an annual calendar . Try it out. Feedback welcome!
Sun Clock is now a Progressive Web App . This means you can install it on your device homepage and it will be available when your are offline.
Added auto-color mode (dynamic colors that change with the time periods.)
Added dark mode.
Live!
GitHub has been having enough outages lately that people are starting to ask whether it makes sense to look elsewhere.
Ever since Github got acquired by Microsoft it started getting bloated and enshittified - it lost its direction and doesn't seem to care about its customers. And Github is probably not the only one - there are many other examples.
We got to have a kill switch for this - when do we as a community say enough is enough and switch from GitHub. If a tool constantly proves unreliable, keep using it is not logical.
I really hope Github pulls up its socks and really addresses the problems - the network effect will not last for long if you keep having outages.
As pointed out pertinently on hn - "But there isn’t one obvious transition candidate, so the diaspora is finding itself spread across a bunch of disparate places and services. The stated need they satisfy (source control with a web interface and some technical features, like pull requests with reviews) will be fulfilled. But those emergent features like a core community and default expectation of where you can find someone will fade. And that is a very real loss for all of us."
If you are already an LWN.net subscriber, please log in with the form below to read this content.
Please consider subscribing to LWN . An LWN subscription provides numerous benefits, including access to restricted content and the warm feeling of knowing that you are helping to keep LWN alive.
(Alternatively, this item will become freely available on August 27, 2026)
TL;DR: DuckDB v2.0 is coming this fall. In this post, we preview its headline features: DuckDB as a server, triggers, the VARIANT type, asynchronous I/O, a new SQL parser, a new storage format, and much more.
DuckDB v2.0 will be named “Cyanoptera” after the cinnamon teal (Anas cyanoptera), a strikingly reddish-brown duck found in the western Americas.
A major version bump is not something we do lightly, and it is not just ceremony: v2.0 ships a new SQL parser, a new default storage format, a reworked C API, and a small number of carefully chosen breaking changes. But above all, it is a feature release, built from over 10,000 commits since we released v1.5 in March. Where last year was the year of the lakehouse, this release kicks off the year of DuckDB as a server. We previewed many of these features in the “State of the Duck” talk at DuckCon #7 , if you prefer to watch instead of read.
DuckDB is moving rather quickly, and we can only cover a small fraction of the changes here. Condensing all new features down to a shortlist is always a fight over what gets in, and yes, we know that what follows is technically a listicle (Ten Things Coming to DuckDB v2.0, Number Eight Will Shock You). We are not proud of the format, but it works, so here it is, starting with the SQL-level features and working down into the engine.
CONNECT
DuckDB has been an in-process database since day one. But people have asked us – very persistently – for a client/server mode, and we have finally caved. The
quack
extension
implements DuckDB's native protocol for talking to other DuckDBs. It was
released as a preview
shortly before DuckCon #7, graduates to stable in v2.0, and it is a big part of where DuckDB is headed: any DuckDB process can serve its databases over the network, and any other DuckDB can attach to it and route queries there using the new
CONNECT
statement. For example:
CALL quack_serve(
token = 'my_token'
);
quack:
ATTACH 'quack:server.example.com'
AS qk (TOKEN 'my_token');
CONNECT qk;
SELECT count(*) FROM events;
-- executes on the server,
-- results stream back
DISCONNECT;
CONNECT
is the successor to the
remote.query($$...$$)
workaround we showed when Quack was first revealed – we looked at that syntax and said: no, this cannot be it. And
CONNECT
is not limited to Quack: it points your session at any remote database that supports it, and the new remote pushdown optimizer (
#22914
) ships SQL directly to PostgreSQL and MySQL instead of pulling tables over the wire:
CONNECT 'postgres://localhost/mydb';
SELECT count(*) FROM orders; -- runs on the PostgreSQL server
DISCONNECT;
If you have worked with analytical systems in the past, you may assume that DuckDB cannot handle transactional workloads. But DuckDB has been built as a transactional, multi-connection database with full MVCC and transaction isolation since day one. Most users just never needed that in a single-user scenario. It turns out DuckDB handles transactions well: it's fast enough to compete with general-purpose databases like PostgreSQL on quite a few workloads, and the client/server pattern finally lets that machinery shine in multi-tenant, long-running deployments.
Running DuckDB long-term also comes with new challenges, which is why v2.0 pushes on better metrics, logs, and observability (see, e.g., the metrics layer rework in #22799 ) that let you look at a DuckDB instance and see what it is actually doing. People even built standalone clients for the Quack protocol within weeks of the preview. We thought we were extending DuckDB to talk to other DuckDBs; the world said no, no, no, and built their own clients. Who would have thought.
VARIANT
Becomes a First-Class Citizen
The
VARIANT
type shipped in
DuckDB v1.5
, and the way to think about it is JSON on steroids. Basically, imagine if JSON were fast. Like JSON, a
VARIANT
column can store differently-shaped data in every row. Unlike JSON, it is not a text format: DuckDB automatically detects the common structure hidden in your semi-structured data and “shreds” it, so it compresses well in storage and executes fast in queries, all without you ever declaring a schema. This makes
VARIANT
a natural fit for real-time log ingestion, where streams of JSON-ish records share structure but evolve over time.
In v2.0, this pipeline works end to end: shredded execution straight from storage (
#20912
), extraction pushdown into scans (
#22478
), shredded
VARIANT
reading
and
writing for Parquet, and a family of
variant_*
functions:
CREATE TABLE events (payload VARIANT);
INSERT INTO events
VALUES ('{"user": {"id": 42, "tags": ["a", "b"]}}'::JSON::VARIANT);
SELECT variant_type(payload), variant_keys(payload)
FROM events;
SELECT *
FROM events
WHERE variant_contains(payload, {'user': {'id': 42}}::VARIANT);
Longer term, likely soon after v2.0 (but don't hold us to it), we plan to back the regular
JSON
type with
VARIANT
, so existing JSON workloads get all of these benefits without changing a single query.
Triggers have been a long-standing feature request, and DuckDB v2.0 delivers them in full:
BEFORE
and
AFTER
triggers,
FOR EACH ROW
and
FOR EACH STATEMENT
, transition tables via
REFERENCING OLD/NEW TABLE
, multiple triggers per event,
RETURNING
on triggered tables, and
DROP TRIGGER
.
The classic use case is audit tables: something happens in the system, and a trigger records what changed. For example:
CREATE TABLE target (id INTEGER, val INTEGER);
CREATE TABLE audit (id INTEGER, old_val INTEGER, new_val INTEGER);
CREATE TRIGGER trg_audit AFTER UPDATE ON target
REFERENCING OLD TABLE AS o NEW TABLE AS n
FOR EACH STATEMENT
INSERT INTO audit
SELECT n.id, o.val, n.val
FROM o
JOIN n ON o.id = n.id;
INSERT INTO target VALUES (1, 10), (2, 20);
UPDATE target SET val = val * 10 WHERE id <= 2;
SELECT * FROM audit;
| id | old_val | new_val |
|---|---|---|
| 1 | 10 | 100 |
| 2 | 20 | 200 |
Triggers fit naturally with long-running DuckDB services, and we are also planning to use them internally to build several upcoming features. They are fully exposed at the SQL level too, so you can build your own cool stuff with them.
As always, DuckDB's SQL dialect keeps growing. A few favorites from this release cycle:
With
NEAREST
joins
(
#24137
), top-k similarity search becomes a join clause, handy for vector and embedding workloads:
SELECT q.user_id, t.product_id
FROM users q
INNER JOIN products t APPROX NEAREST 2
BY SIMILARITY array_cosine_similarity(q.embedding, t.embedding);
DML inside CTEs
(
#21634
,
#21997
,
#24217
) lets you use
INSERT
,
UPDATE
,
DELETE
, and
COPY
as pipeline steps:
WITH moved AS MATERIALIZED (
DELETE FROM staging RETURNING *
)
INSERT INTO archive SELECT * FROM moved;
Nested schemas ( #23492 , #24222 ) allow schemas within schemas:
CREATE SCHEMA finance;
CREATE SCHEMA finance.reports;
CREATE TABLE finance.reports.q3 (revenue DECIMAL);
The new
variable syntax
(
#21194
) lets you write
$x
anywhere an expression is allowed, no more
getvariable(...)
verbiage
:
SET VARIABLE threshold = 100;
SELECT * FROM orders WHERE amount > $threshold;
The
JSON mutation functions
json_set
,
json_insert
,
json_replace
, and
json_remove
(
#23786
) finally let you modify JSON documents in place:
SELECT json_set('{"a":1}', '$.b', '2');
| json_set('{"a":1}', '$.b', '2') |
|---|
| {"a":1,"b":2} |
And
recursive CTEs with
USING KEY
aggregation
(
#19481
) enable iterative algorithms in pure SQL, backed by the rewritten recursive CTE engine described below:
WITH RECURSIVE tbl(a, b) USING KEY (a, avg(b)) AS (
SELECT 1, 5
UNION
SELECT a, b - 1 FROM tbl WHERE b > 0
)
TABLE tbl;
| a | b |
|---|---|
| 1 | 2.5 |
There is more: SQL-standard
FETCH FIRST 2 ROWS ONLY
(
#23533
),
OVERLAY()
(
#22456
),
UNNEST
in
GROUP BY
(
#23644
), and well-defined
MERGE
/
UPDATE ... FROM
semantics for multi-matched rows (
#24058
).
Interacting with object stores like S3 is central to the DuckDB experience: your data has to come from somewhere, and it often sits in object storage. DuckDB has long been able to read from object stores in parallel, but synchronous access placed a limit on how fast this could go. DuckDB v2.0 introduces asynchronous I/O throughout the engine. We described the design in detail in a dedicated blog post .
Thanks to asynchronous access, the I/O layer now scales independently from the query processing layer, which means far more parallelism for remote reads and dramatically faster queries on network storage. Parquet support came first (
#23662
), with CSV (
#23961
) and DuckDB's own file format (
#24654
) following, along with asynchronous Parquet writes (
#23283
) and new
MMAP
and
DIRECT_IO
modes (
#22988
). Local storage benefits a little too, but network storage is where you will see the big gains.
As with every release, a lot of work went into making your existing queries faster without you doing anything. To pick some highlights: partial aggregates are now pushed below joins ( #22572 ) and redundant aggregations are reused ( #24543 ), the recursive CTE engine has been rewritten ( #22211 ), aggregations now spill to disk when they outgrow memory ( #24499 ), and the Windows CLI got approximately 2.2× faster at multi-threaded result materialization ( #24036 ).
How much faster can this get? Here is a microbenchmark you can run on a laptop: single-source reachability over a graph with one million edges, written as a plain recursive CTE .
CREATE TABLE edges AS
SELECT (range % 100_000)::INTEGER AS src,
((range * 13 + 7) % 100_000)::INTEGER AS dst
FROM range(1_000_000);
WITH RECURSIVE reachable(node) AS (
SELECT 0
UNION
SELECT dst FROM edges, reachable WHERE src = node
)
SELECT count(*) FROM reachable;
| Version | Run time |
|---|---|
| DuckDB v1.5.4 | 4.90 s |
| DuckDB v2.0 (preview) | 0.12 s |
As you can see, DuckDB v2.0 is about 40× faster (!) for the same recursive query.
Row-group pruning has been massively expanded:
min-max indexes (zone maps)
and
Parquet Bloom filters
now skip data for structs, lists, decimals, UUIDs,
IN
filters, and even function predicates:
-- these now prune row groups instead of scanning them:
SELECT * FROM logs WHERE contains(message, 'ERROR');
SELECT * FROM t WHERE substr(code, 1, 3) = 'NL-';
SELECT * FROM 'data/*.parquet' WHERE id IN (1, 5, 9);
Query planning also becomes partition-aware ( #22336 ). Lakehouse formats (DuckLake, Iceberg and plain Hive-partitioned Parquet on S3) are all partitioned, and exploiting that partitioning is often the difference between scanning a dataset and skipping most of it. In v2.0, the planner and optimizer take full advantage of existing partitioning, and partitioned writes have been reworked as well ( #22225 , #22620 ).
DuckDB v2.0 bumps the default storage format version to v2.0.0 ( #22875 ). The headline change is buffer-managed ART indexes ( #21458 , #23605 ): indexes are no longer pinned in memory, which means large indexed tables open instantly and their indexes are paged in on demand.
Column metadata is now loaded lazily (
#22333
), so wide tables open faster too. The
DICT_FSST
string compression method is enabled by default (
#23733
), deletes are stored compactly (
#24336
), and the storage layer performs much stronger corruption validation on read. In short: databases with big indexes and wide tables open faster and use far less memory.
DuckDB has famously always used a parser derived from PostgreSQL's. We have decided that enough is enough: v2.0 ships our own modern, extensible PEG-based parser ( #22194 ), an idea we first explored in our 2024 post on runtime-extensible parsers . This change ties into the extension ecosystem: extensions can now hook into the grammar itself, so expect extensions that expose entirely new SQL syntax. It also brings better error messages with precise source locations, and the first dialect compatibility mode:
SET dialect_compatibility_mode = 'spark';
You should not actually notice anything from the parser swap as we designed it to be compatible with the old one. If you do notice, please file an issue.
Timezone-aware timestamps, calendars, and collations in DuckDB have always been powered by the ICU library. ICU is a fine library, but we only ever used a small slice of it, while still carrying it around in every DuckDB distribution. In v2.0, the ICU library is gone entirely: the
icu
extension now implements timezones, calendars, and collations itself (
#24463
,
#24403
), with the timezone data built directly from the IANA database and compressed down to around 45 kB. Everything keeps working exactly as before:
SELECT '2026-08-14 12:00:00'::TIMESTAMPTZ AT TIME ZONE 'Europe/Paris';
SELECT * FROM names ORDER BY name COLLATE de;
Besides being much smaller and easier to keep up to date, the new implementation is also simply faster. Here's a quick microbenchmark on a MacBook that converts 25 million timestamps to a timezone and filters 5 million strings with a German collation:
| Query | v1.5.4 (ICU) | v2.0 (native) | Speedup |
|---|---|---|---|
ts AT TIME ZONE 'Europe/Paris'
, 25 M rows
|
0.24 s | 0.11 s | 2.2× |
Filter with
COLLATE de
, 5 M rows
|
0.15 s | 0.06 s | 2.6× |
Extensions are one of the best things about DuckDB, but today, most of them, including our own, build against the unstable C++ API . That means extension authors have to re-target and rebuild for every DuckDB release, and community extensions can silently disappear when their authors stop keeping up. DuckDB v2.0 broadens the stable C API far enough that extensions can be written once, built once, published once, and keep working, essentially until the end of time.
To make this sustainable over the long run, the C API is now generated from a declarative, versioned specification (
#24135
): every function in
duckdb.h
,
duckdb_extension.h
, and the extension ABI is described in YAML in the
api_spec/
directory
, with its full lifecycle on record, and CI verifies the committed headers against the spec so API and ABI can no longer drift apart. The release also brings unified symbol versioning (
#24435
), custom allocation handlers (
#23945
), and static linking of C API extensions into your application (
#22251
).
So what does building an extension against the stable C API look like? Here is a complete extension: a single file that registers a vectorized scalar function, compiled once against
duckdb_extension.h
.
#include "duckdb_extension.h"
DUCKDB_EXTENSION_EXTERN
// a scalar function that adds two BIGINTs, one vector at a time
static void AddNumbers(duckdb_function_info info, duckdb_data_chunk input, duckdb_vector output) {
idx_t count = duckdb_data_chunk_get_size(input);
int64_t *a = (int64_t *) duckdb_vector_get_data(duckdb_data_chunk_get_vector(input, 0));
int64_t *b = (int64_t *) duckdb_vector_get_data(duckdb_data_chunk_get_vector(input, 1));
int64_t *result = (int64_t *) duckdb_vector_get_data(output);
for (idx_t row = 0; row < count; row++) {
result[row] = a[row] + b[row];
}
}
DUCKDB_EXTENSION_ENTRYPOINT(duckdb_connection con,
duckdb_extension_info info,
duckdb_extension_access *access) {
duckdb_scalar_function f = duckdb_create_scalar_function();
duckdb_scalar_function_set_name(f, "add_numbers");
duckdb_logical_type bigint = duckdb_create_logical_type(DUCKDB_TYPE_BIGINT);
duckdb_scalar_function_add_parameter(f, bigint);
duckdb_scalar_function_add_parameter(f, bigint);
duckdb_scalar_function_set_return_type(f, bigint);
duckdb_destroy_logical_type(&bigint);
duckdb_scalar_function_set_function(f, AddNumbers);
duckdb_register_scalar_function(con, f);
duckdb_destroy_scalar_function(&f);
return true;
}
LOAD add_numbers;
SELECT add_numbers(40, 2);
For brevity, we skipped
NULLhandling here. See thedemo_capiextension for the full version.
The binary this compiles to keeps working across DuckDB versions. You do not need re-target or rebuild it every time a new DuckDB version comes out. And nowadays, with all the AI tooling around, building an extension has never been easier.
So you have written your extension. But how should you distribute it? Until now, DuckDB could only install extensions from the built-in repositories (
core
,
core_nightly
,
community
, …). In v2.0, you will be able to register your own trusted repositories (
#24777
, currently work-in-progress), so an organization can host and sign its own extensions and have them install and load just like the built-in ones:
SET allow_extension_repositories = 'allowed';
CREATE EXTENSION REPOSITORY my_repo FROM 'https://extensions.example.org';
INSTALL my_ext FROM my_repo;
LOAD my_repo/my_ext;
A repository is a name, a URL prefix, and one or more RSA public keys that are trusted to sign the extensions served from it. The prefix can point at anything DuckDB can read: a local path,
https
,
s3
, you name it. At
CREATE
time, DuckDB fetches the repository's public keys and pins them into the repository definition, printing each key's SHA-256 fingerprint so you can compare it against one published out of band. If you would rather not trust the network at all, you can pass the key directly:
CREATE EXTENSION REPOSITORY my_repo FROM 's3://my-bucket/extensions'
USING PUBLIC KEY '-----BEGIN PUBLIC KEY----- ...';
Pinned repositories survive restarts, support key rotation by trusting multiple keys, and can be audited at any time through the
duckdb_extension_repositories()
table function, or removed again with
DROP EXTENSION REPOSITORY
. Together with the stable C API, the extension story rounds out nicely: write your extension once, sign it, host it wherever you like, and
INSTALL
it anywhere.
Starting this fall, we will add a stakeholder advisory board to the DuckDB Foundation . The advisory board will provide input on the development roadmap of DuckDB, DuckLake, and Quack. This allows key stakeholders to have a say in the projects' direction.
These are only a few highlights, and this post is only a preview. Some details may still shift before the release this fall, and there are many more features and improvements that we could not cover here. DuckDB v2.0 will also come with a small set of breaking changes, including the new default storage format and the completed lambda syntax transition, which we will cover in detail in the release announcement.
There have been more than 10,000 commits by many contributors since we released v1.5. We would like to thank our community for the detailed issue reports, feedback, and contributions that shaped this release. If you want a taste before the fall, the preview builds have most of these features today, and if something breaks, you know where the issue tracker is.
Pedro Holanda
If you are already an LWN.net subscriber, please log in with the form below to read this content.
Please consider subscribing to LWN . An LWN subscription provides numerous benefits, including access to restricted content and the warm feeling of knowing that you are helping to keep LWN alive.
(Alternatively, this item will become freely available on August 27, 2026)
A Denver District Court judge set a process for Nine PBS to retrieve its archival data and programming from Iron Mountain Data Centers during a Wednesday hearing.
The St. Louis station sued the information management company July 28 in Denver District Court seeking the return of approximately 50 terabytes of archival material stored in a Denver data center.
The complaint alleged that Iron Mountain refused to return Nine PBS’ materials because OSS, the now-defunct company that the station had contracted with for storage services, technically owned “the physical services housing” the station’s data within Iron Mountain.
District Court Judge Eric Elliff ordered Iron Mountain, which had a separate business relationship with OSS, to cooperate with Nine PBS in any way possible to retrieve the data. He found that the station is the rightful owner of the materials and entitled to recover them from OSS’ storage systems.
Under his order, Nine PBS is to identify a third-party vendor, such as a former OSS employee, who can assist in accessing and retrieving the data from the infrastructure that’s housed in Iron Mountain’s center within 30 days.
Elliff acknowledged the complexities of Iron Mountain’s position as a vendor to OSS, which, according to Nine PBS’ complaint, is in delinquency. Iron Mountain is the “custodian” of Nine PBS’ data, but it isn’t the vendor that contracted with the station to store and preserve its data. That obligation remains with OSS. Under the order, Nine PBS will pay Iron Mountain current and past-due fees for data storage, starting from when OSS stopped paying Iron Mountain for use of its data storage facility.
In a statement to Current, Nine PBS VP and CCO Leah Freeman expressed gratitude to the court for providing a framework for the data retrieval.
“We appreciate the Court’s thoughtful decision establishing a path forward to access and recover our archival materials, which the Court confirmed that Nine PBS rightfully owns,” Freeman said. “These archives represent an important part of our region’s history, and we look forward to ensuring their preservation and protection through the Court-approved process.”
During the hearing, Gregory Rich, an attorney representing Nine PBS, said the station seeks access to a physical cage where the data is housed within Iron Mountain’s facility. The station is in contact with a former OSS employee who is willing to help obtain the data. The attorney noted that the data could potentially be stored in physical form, such as tapes that could be easily retrieved. But if the materials are on a server, Nine PBS could lose the materials forever if Iron Mountain shuts it down.
William Cravens, the attorney representing Iron Mountain, told the judge his client doesn’t know the format of Nine PBS’ materials that were stored by OSS. He expressed concern about whether Nine PBS’ archival material is lumped together with data from other OSS clients. Iron Mountain wants to avoid potentially corrupting the other data, Cravens added.
Elliff ordered the immediate return of any physical devices that hold Nine PBS’ data once access to OSS’ storage system is granted. If data retrieval turns out to be more complicated — if it is encrypted, for example — he will schedule another hearing to determine how to proceed.
Once Nine PBS retrieves its data, the station must work with a third party to ensure that no data from other OSS customers is among those materials.
Nine PBS’ complaint includes a list of the files that make up its 50 terabytes of data. Elliff ordered the parties to compare that list to the files recovered from OSS storage once the data is retrieved.
He also ordered Nine PBS to defend and indemnify Iron Mountain should other data be corrupted in the process of retrieving its archival materials.
The judge ordered both parties to provide updates on the data retrieval process by Sept. 14.
In the previous chapter , we got the data: an LLM-powered pipeline matched 74% of the benchmark functions. Now it’s time to see how it performs on a whole game.
After one year of studying and building tooling for matching decompilation with AI, I finally stopped postponing and started to decompile “ Klonoa: Empire of Dreams ” (KEoD)! It's a Game Boy Advance game that I really love and, as we’ll see, a challenging one: its functions are unusually big.
⚙️ What is Matching Decompilation?
Matching decompilation is the art of converting assembly back into C source code that, when compiled, produces byte-for-byte identical machine code. It’s popular in the retro gaming community for recreating the source code of classic games. For example, Super Mario 64 and The Legend of Zelda: Ocarina of Time have been fully match-decompiled.
At the time I’m writing this chapter, 51% of the game’s code bytes are decompiled. Many thanks to Felipe Sanches and testyourmine for helping to reach this milestone!
Spoiler: We’ll see how Claude Code autonomously found an easter egg that had been hidden for 25 years!
The first step is, of course, scaffolding the project. There is no single standard for how a decomp project should be structured, although there are some tacit structures that the community usually follows. In the case of GBA, one of the most popular is the one used by pret .
My initial idea was to follow the same code organization as Sonic Advance 3 , since I was more familiar with it, but I quickly diverged because I had set the following goals:
It should have no assets or assembly code from the ROM. That's the case with Snowboard Kids 2 and Animal Forest , for instance.
I want to make the setup reproducible, as a paved path that other GBA games will follow, since I want to decompile the other Klonoa GBA games. Building a foundation to replicate later is important.
And more importantly, the project organization should make it easy for an AI agent to work with.
These goals have two important implications:
Setup based on a script:
We need to have an automated script to, given a
.gba
ROM, produce the
.s
files from it, since they aren't versioned. This implies that any change to the
.s
files must be made in the script that generated them, not to the
.s
files directly.
For example, renaming functions or splitting them into modules should be done in the generator script. This diverges from how pret projects work, since the assembly is committed.
As little manual work as possible:
Since this project should be AI-friendly, it should be possible to easily spawn a
git worktree
to enable parallel work, and it should be easy to add a new matched function.
With those goals set, the first artifact is the assembly itself. For that, I used
Luvdis
. It’s a GBA disassembler designed for matching decompilation. It reads KEoD's ROM and outputs a single big
.s
file.
Since I don’t want to have any assembly source code from the original game in Git, I included Luvdis as a git submodule and a shell script to call it,
./setup.sh
. Besides calling Luvdis, this script will grow to do all the transformations we’ll discuss next: compiling the compiler, refining the disassembled code, moving the
.s
files, etc.
We need to find the compiler the developers used. Since many GBA games used agbcc (a fork of GCC 2.95), that's likely the case for KEoD too.
It’s worth mentioning that KEoD was released just 4 months after the GBA itself, in July 2001. It ranks as the oldest GBA game that is actively being decompiled! 1
With this release date in mind, we can rule out two popular compilers used for GBA game development: ADS 1.2, which was released in November 2001 , and Metrowerks’ CodeWarrior, which was released in April 2002 .
Of course, the developers could have been unconventional and used a different compiler, such as ADS 1.1. In any case, I started with agbcc, and since it enabled the match for simple functions with clean C code, I stuck with it. It was a no-brainer. 2
To compile these first simple functions, we needed to have a Makefile and a linker script working. They were written by Claude Code using a few other decomp projects as inspiration.
Although Luvdis is great to start with, we still need to make many improvements to make the disassembled code ready to be decompiled. Let's talk about them.
Luvdis automatically detected 93 functions, but the game surely has way more than that. So, I used Ghidra to enrich the function list, which found ~300 new functions. After using Claude Code to find even more functions, merging the results, and cleaning up the false positives, we ended up with a total of 663 functions .
I was surprised that this game has only a few hundred functions. Other GBA games have many more functions. On the other hand, the functions from KEoD are usually bigger, much bigger. That’ll make the game a challenge to decompile since bigger functions are usually harder than smaller ones.
Most of the function splitting was driven by Claude Code. It wrote a reproducible Python script,
generate_asm.py
, that takes the initial code output by Luvdis and splits it into smaller functions, and each function is in its own
.s
module.
Although most of the work was done by AI, let me briefly explain one interesting technique it used and one of the bugs we had. It took many back-and-forths to arrive at a sound function split, but these two sections will give a glimpse of how it works.
Ghidra found functions that are directly referenced by the code. For example, ones that had a
bl <function>
. But many functions are still called without any
bl
appearing in the assembly. This happens because the code uses a function table. For example:
It compiles into:
So, we need to find the table's address in the literal pool (here,
0x08000100
) and unpack the words stored there into function addresses.
This pattern repeats frequently in this game since it has a big callback machine and many dispatch tables. Take gCallbackQueue as an example. Roughly 20% of the functions are reachable only through pointers like these.
When Claude Code was splitting the functions, it defined the address at which each function starts. But when an address is misplaced, it produces a function fragment. In other words, a function that isn't self-contained. For example:
There is no return in
FragmentedFunction
. Thus, it falls through to the next function, running the code from
FallthroughFunction
. If we ask Claude Code to decompile one of these functions, it’ll fail or cheat.
This happens because the above split is bugged. By merging these mis-split functions, we can write C code that matches the merged version:
It compiles to:
A fragmented function can happen if:
We had a bug in our function-splitting algorithm (almost always the case).
The assembly was handwritten (unlikely)
The original C code used the naked attribute (unlikely)
After splitting the functions, the next step is splitting the monolithic assembly file into logical modules. I used Ghidra to find good boundaries (
see the Ghidra script here
) and then updated
generate_asm.py
to automatically move
.s
modules into their respective folders. The boundaries are defined
in the TOML file
.
The renaming is also performed in
generate_asm.py
, based on the names
defined in TOML
.
Sanches used Claude Code to automatically give semantic names to the functions. Even though they aren't decompiled yet, Claude provides a reasonable guess based on the assembly patterns.
When a function is compiled, in addition to its code, it might include data that it references. For example:
Compiles into:
Without a proper disassembler that identifies that a given address is read as data rather than as code, it would be disassembled as:
Although the content is the same and both compile to byte-identical machine code, this spelling is bad for many reasons:
Programmatic decompilers like m2c and asmlift produce wrong C code or decline, since they’ll try to convert the nonsense instructions into C code. LLMs also struggle for the same reason
objdiff , the tool we use to count how closely our C code matches the target assembly, counts data as mismatched instructions. It inflates the diff and makes it harder to identify real code differences
Remember the previous section “Function pointer table”? Well, without seeing the actual function addresses, there is no way to find the pointer table in the assembly
Luvdis already does a fair amount of work recovering every pool it can prove is data, but it misses a few cases that are relevant when we want to fully decompile the game. Thus, I ran a couple of rounds asking Claude Code to identify the data-as-code sections and save them
in the TOML file
. It's consumed by
generate_asm.py
to update the assembly code.
We need to have a smooth process to replace an assembly function with its respective decompiled C code. One function at a time.
So, I structured it with two folders:
asm/matchings
and
asm/nonmatchings
. When a function is matched, it’s automatically moved to
asm/matchings
. By always preserving the original assembly functions (and not relying on the compiled ones from
build/src
), we have a single assembly spelling.
Otherwise, we would have the spelling from
generate_asm.py
for the non-matched functions and the spelling from agbcc for the matched ones. Ensuring that we have a single one helps reduce the mental gymnastics when comparing the functions, and it's especially important when Claude Code is reading through the codebase.
Also with Claude Code in mind, and inspired by Snowboard Kids 2, I organized the C code with the
INCLUDE_ASM
macro. Each function that isn't decompiled yet appears as a stub like:
This way, the agent can easily add a new matched function: normally, it just needs to replace the
INCLUDE_ASM
call with the equivalent C code.
Later, I learned that this approach using the
INCLUDE_ASM
stub is informally named “splat-style”, since that's how
splat
scaffolds the project. This tool splits a binary into source files that are easier to work with. It supports N64, PSX, PS2, PSP… but not GBA. Thus, funnily, I’m replicating the splat-style without splat.
After I matched a set of simple functions, testyourmine created his own repository to decompile this game ( kleod ). He was willing to contribute but didn't like my project's structure, and my code was too messy (I agree 😅).
So, I imported most of the functions he decompiled and part of the project structure. Even though the underlying structure is different, it's good to follow some community standards, since it’ll make it easier to share code and get inspiration.
For example, using the same names for the m4a functions and utility macros common to every GBA game (e.g., the ones for DMA) helps reduce the variance in the codebase and reduce the cognitive overhead when reading the code from other GBA decomp projects while looking for references.
That matters for LLMs: Claude Code reads the code from them to find references, and reducing the discrepancy allows one to make better use of these references.
On top of that, his code quality is great. His C code is much cleaner than the AI-generated one. Porting his functions to my repository sets a better standard for Claude Code to follow when decompiling other functions.
The hard work of decompilation is mostly done by Claude Code. But just asking it to decompile and hoping it does everything well is expecting too much from AI.
So, let's talk about what we're going to plug into the AI loop to make it work: new tools and improvements to agbcc.
As you probably already noticed, I really like to build tools, and aiming to accelerate the decompilation process, I’ve grown the family! The following tools joined the party: asmlift, Transmuter and gba-kit.
Each one would deserve a dedicated chapter, but let's talk briefly about them.
The driving question that led to building asmlift is: What if, instead of making a match, we build a machine that does the match for us?
asmlift is a programmatic decompiler. It's much like m2c . In fact, much of the learning from m2c was used to build asmlift. The twist is how it's developed.
Most of the matching decompilation projects heavily based on AI focus on matching a single function at a time. They ask the AI to match a function, and as an additional output, it might produce markdown files with learnings, hoping these will help the LLM to decompile the next functions.
But I was keen to explore a different approach: using AI to design, from scratch, a modular matching decompiler, plus using an AI loop to automatically iterate on the decompiler itself.
asmlift is the result of this exploration. While developing asmlift, I’m much more focused on the benchmark and evaluation side than on the decompilation process itself.
Curiously, while I was building asmlift, mahaloz was building Kuna , an agent-first decompiler designed to be refined by other agents. You can read more about Kuna in its announcement post . It's interesting to see that the same broad idea was developed by two people independently, even though the decompilation scene isn't big. It's worth noting the difference: asmlift is designed for matching decompilation and focuses on the compilers used by retro games, while Kuna is a more generic decompiler, and the matching is just one of its metrics, not the primary one.
I’m using asmlift while decompiling KEoD, and it's been very useful. Sometimes, it even matches a non-trivial function like ProcessFrameAnimation in one shot. You can run its playground and check the benchmark here .
One of the most widely used decomp tools is Permuter . I already mentioned it in the last 2 chapters, but just a refresher: it programmatically permutes C source code to find a better match.
But I really wanted a coding-agent-friendly permuter . Instead of mostly random permutations that Permuter makes, I want to have Claude Code actively monitoring, guiding the process, and testing hypotheses.
Developing a new spiritual successor to Permuter seemed to be a suitable approach. I believe that, to achieve this, it would need to be rebuilt from scratch with a new design. Besides that, it wasn’t actually a novel idea, since Simon, the creator of Permuter, suggested it on Discord.
So, I started (vibe) coding Transmuter . I also chased the following stretch goals:
Library-first design. I wanted to have deeper integration into Mizuchi (the AI decompilation platform from the previous chapter).
Multi-language support. Permuter supports only C, since it’s heavily dependent on pycparser. Since I have good experience with ast-grep, adding multi-language support sounds like a good addition.
Post-match cleanup. It’s something that I miss: if I have code that already matches, but it’s ugly, I want to clean it up, and having quick programmatic mutations can speed it up.
Web app for session reports. Why not? It’s cool and helps to understand what’s going on when debugging or running it manually. Also, the same data that feeds the session report can be read by coding agents, offering more input for future work.
Although Transmuter works, it’s still an early-stage tool. I’m dogfooding it while decompiling KEoD and sometimes it helps find a match, though most of the time it doesn't provide any useful insights. It might become a more useful tool in the future.
gba-kit is a GBA emulator focused on scripting. I initially started to develop it to explore behavioral decompilation, as explained briefly in the last chapter. But then I noticed that it's useful as a programmatic emulator that an AI agent can use to explore the game and make discoveries . Then, I threw away all the features for behavioral decompilation and focused only on the scripting side.
For matching decompilation, the main use case is improving the code quality using runtime analysis. For example, asking the coding agent to explore the game and replace an unknown property with a meaningful name. The same goes for functions, their parameters, constants, etc. 3
At the beginning of the tests, it was funny watching Claude Code explore Klonoa. During my first experiments, I just asked it to finish the first level without giving it any instructions about how to play. Here are three fun anecdotes:
It thought that the stars required to finish the level were locked behind breakable blocks and that it needed to throw enemies at them.
Claude screamed “MASSIVE PROGRESS”, “SCREEN TRANSITION!” and “COMPLETELY NEW AREA!” when it saw a black transition, thinking that it had moved to a new area. But it was actually just Klonoa dying after being hit by an enemy three times.
It thought that Klonoa was a Sonic-like game. It said “ Reading 0x0300526C in real gameplay and watchpointing it confirmed the field is live and that it’s decremented by the gameplay VBlank handler when Klonoa takes damage (0x800d2c2) — matching Klonoa’s signature mechanic of dropping a dream stone when hit.”
After these initial fun conversations, I taught it how to play the game, giving it recorded scripts as examples, and it learned quickly and wrote many scripts to evaluate some scenarios. The first contributions were adding meaningful names for the properties of a global game-state struct.
Additionally, a big win was finding an easter egg that had been hidden for 25 years! Nobody had found it before: you can play a minigame on the “Clear Save Data” screen. Surprisingly, it was found autonomously by Claude Code. When it claimed that it had found an easter egg, I initially thought that it was hallucinating, but it turned out to be real.
macabeus @bmacabeus
Found an easter egg in "Klonoa: Empire of Dreams" while decompiling it with Claude Code ✨ Press Select on the "Clear Save Data" screen, and Klonoa appears in the middle of the screen. Now, dodge the falling Moos!
10:16 PM · Aug 1, 2026 · 8.25K Views
4 Replies · 21 Reposts · 136 Likes
I’m decompiling KEoD while improving these tools. The core of the flow is:
Ask Claude Code to select X functions to decompile.
Use asmlift and, if it doesn't match automatically, continue from its result. Use Transmuter to help find the match.
After matching the function, use gba-kit to improve the code quality.
Launch adversarial subagents to review the code.
Merge all the X decompiled functions into a single PR.
Write a report.
I hope to turn it into a self-improving AI loop. But for now, I still need to read the report to guide the improvements.
You can check the prompt for my AI loop, wiring these tools to decompile the functions from KEoD (it’s in the asmlift repository for historical reasons). Note that this prompt is tailored for my use, but it might be useful as inspiration for you.
You may notice Mizuchi is absent from that loop. That's deliberate: its inner loop and test environment, which are useful for isolating the execution to make the runs comparable with each other, are troublesome when contributing to a decompilation project.
I still used Mizuchi for benchmarking; for example, I was curious about Fable 5 and benchmarked it . But to be a participant in this loop, I would need to extract a few features from it, mostly the ones from Atlas (e.g., function embeddings to find similar functions), so they can be reused independently and called autonomously by a coding agent. That's something that I’m planning to do in the future.
agbcc is our oracle. We want to have C code as close as possible to what the developers originally wrote. Thus, you'd expect us not to change the oracle, right? Well, it depends™️
Let's see one case where we shouldn't, and one where it's fair.
I faced an annoying problem at the beginning of the project. When the AI agent gets stuck decompiling a function, it’ll start trying to cheat, for example, by forking agbcc.
At first, I believed that the fork was justified ( PR 1 ), since it was plausible: KEoD is one of the first GBA titles, and I believed that it could have used an older, unknown version of the compiler. But it was just me having AI psychosis, and it was not necessary at all .
The cheats applied by Claude Code, like forking agbcc or relying too much on register pins, which it justified by saying that the compiler ordered the registers in a way that's “impossible” to reproduce in C, were quickly dismissed by more experienced decompers.
On the other hand, there is a fair and safe case for customizing the compiler: to add instrumentation that Claude Code can read.
I asked Claude Code to improve the code quality (e.g., remove
asm
barriers, register pins, and orphan blocks), and during the process I noticed that adding instrumentation to the compiler would be useful and more viable than adding ad-hoc logs that would be removed later.
Thus, Claude Code added a couple of flags that don't change the compiler's output semantics but only add comments in the assembly or output to stderr.
For example, the flag
-finstrument-src-locs
emits
@ src:file.c:LINE
asm comments, which is useful for backtracking to the C code by just reading the assembly. These new flags were helpful, mostly to improve the code quality.
You can check the fork here .
Parallel to everything above, Jack Price-Burns released Decomp Academy . This website fills a gap: a structured way to learn the basics. It made a big splash when published and ranked in the second position on Hacker News !
Jack initially made it only for GameCube, focusing on Star Fox Adventures. But I think it's so cool that I pushed a commit adding Game Boy Advance lessons! The GBA course is still very raw, and contributions are welcome.
As Jack explained on Hacker News, learning the fundamentals is essential so you don't rely only on AI, since it might hit a wall and be unable to finish the last mile without proper guidance.
Although this game is 51% decompiled, the second half will include harder functions, the ones that take the longest to decompile. Also, although we have many functions decompiled manually by testyourmine, and gba-kit helped to add meaningful names for many of them, the code still needs a cleanup. For example, the comments are far too verbose, and the code needs to be split into semantic modules.
Beyond that, I think the main risk when working on a project heavily reliant on AI is the cognitive debt reaching a level that is impossible to pay off. I’m still learning (as are likely almost all developers) how to enjoy the speed-up that LLMs offer without taking on too much cognitive debt.
An instance of this issue was my initial reaction to the AI cheating that I mentioned earlier. On the other hand, Decomp Academy is an example of investing in the fundamentals to reduce that debt.
Since the biggest risk in this experiment is going insolvent on cognitive debt, the next chapter will probably focus on fundamental learnings .
Still, it's wonderful to see the current results and the foundation that's being built to help other matching decompilation projects. When I started this journey, I had no idea how fun this challenge would be, nor that we would actually be able to automatically decompile many functions purely using AI.
See you in the next chapter!
A single-payer universal health care system could cover every American, save more than 100,000 lives a year, and still cost $1 trillion less than the system it would replace, according to a new preprint study led by researchers at the Yale School of Public Health.
For the study , which has not yet been peer reviewed, the researchers modeled what would happen if the United States adopted a national public insurance program like the one proposed in the Medicare for All Act . Using 2024 spending, insurance coverage, and mortality data, they estimate that the universal coverage would reduce annual health expenditures by $1.04 trillion, or nearly 20% — even after accounting for the additional care that uninsured and underinsured people would receive.
Healthcare costs have been rising faster than inflation, and a staggering share of that spending is consumed by administrative middlemen, soaring drug prices, and emergency care for conditions that should have been treated earlier and for less.
Alison Galvani, PhD
Burnett and Stender Families Professor of Epidemiology (Microbial Diseases) and Director, Center for Infectious Disease Modeling and Analysis
“Healthcare costs have been rising faster than inflation, and a staggering share of that spending is consumed by administrative middlemen, soaring drug prices, and emergency care for conditions that should have been treated earlier and for less,” said senior author Alison Galvani, the Burnett and Stender Families Professor of Epidemiology at YSPH and the director of the Yale Center for Infectious Disease Modeling and Analysis. “Medicare for All strips out those sources of waste while providing everyone with healthcare, saving over a trillion dollars and 114,000 lives every year.”
The model identified five major sources of savings: lower pharmaceutical prices, Medicare-level payments to providers, reduced administrative overhead, less fraudulent billing, and fewer avoidable emergency department visits and hospitalizations. Even under more conservative assumptions about drug prices and fraud reduction, the researchers projected that the savings would reach at least $663 billion a year. Both figures account for an estimated $304 billion in additional spending to meet unmet medical needs, reimburse care that now goes unpaid, and provide universal dental coverage.
A universal healthcare policy would also save tens of thousands of lives, the researchers project. They estimate that adequate coverage for all could avert about 62,863 deaths annually — and that nearly half, or 29,631, would be among people who already hold insurance. These are the underinsured: the more than 45 million working-age adults whose deductibles and cost-sharing put care beyond financial reach anyway. Reversing coverage rollbacks and other health policies enacted since 2025 would avert a further 51,311 deaths each year, the researchers project, bringing the annual total to 114,174.
The study builds on findings Galvani, co-author Meagan Fitzpatrick, and other colleagues published in The Lancet in 2020 , which projected that universal health care would save $450 billion and more than 68,000 lives annually. The larger estimates in the new study reflect ballooning health expenditures, a widening gap between commercial and Medicare payment rates, and new estimates on recent policy changes and the underinsured.
The authors caution that direct estimates of excess mortality among underinsured adults are unavailable, requiring them to model that risk. Their spending analysis also does not account for transition costs, administrative job losses, or how providers might respond to Medicare payment rates.
A system that covers everyone, costs $1 trillion less, and averts more than 100,000 deaths annually requires no new discovery to implement, only enactment.
Even with those limitations, the researchers argue that the United States already spends enough to provide universal coverage. The problem, they conclude, is how that money is allocated.
“A system that covers everyone, costs $1 trillion less, and averts more than 100,000 deaths annually requires no new discovery to implement, only enactment,” they wrote.
The study’s other authors are Abhishek Pandey, senior research scientist in epidemiology (microbial diseases); Chad Wells, postdoctoral research associate; and Yang Ye, associate research scientist in epidemiology (microbial diseases), at YSPH.
Hi HN! I'm Bek, founder of Speko, a platform that finds an optimal combination of speech-to-text, LLM, and text-to-speech models, given your constraints, among all our public benchmarked options, and tells you why.
Demo: https://youtu.be/no2LY2gRh-c
Typical production voice agent is an ensemble of three models: STT, an LLM, and TTS.
Each of those layers offers a dozen credible vendors, and each month there are new models on the market. Almost everyone evaluates once, picks a stack of their choice, and never rechecks because switching from a vendor to another involves yet another integration and arguments about the numbers.
The result is that you use voice agents running last quarter's models while better and cheaper options are available.
Before founding Speko, I spent four years as cofounder and CTO building voice agents for enterprises across Asia in 10+ languages. Each time a new speech model would arrive, we repeated the same ritual: hire native-speaking raters, benchmark it against our existing stack, and update production if it improved. Speko turns this process into an API. A team running thousands of calls a day told us: "we can literally go to this dashboard, switch the model, and it will do it for us."
How it works: you send a request with your optimization criteria (accuracy, latency, cost or balanced), language and region. The router filters to models which we measured for the given combination of constraints, benchmarks them, selects the winner, and returns a response with headers containing provider, model names, and the scores. The gateway prefetches signed session plans, so a new session dials the provider straight from memory; no control-plane round trip while a caller waits.
Failover happens only during connection setup stage: if the provider refuses the connection attempt, we start connecting to the runners-up.
Some of the customer stories: one founder came to us not knowing what to pick at all: he gave us his use case and now routes everything through the platform. A property management AI runs LiveKit in Python and had not updated STT or TTS since launch: they did not know their STT had high error rates on their calls, better options existed, and swapping always looked like an R&D project. One team did not know which models to pick for Spanish. A medical team did not know which STT handles medical vocabulary best. In every case we helped find the right stack from the benchmarks, and now they route through us.
The measuring part is public: we pass the same inputs to every model in one region in different dated runs and we publish the boards, including those where our selections perform worse than alternatives. A launch demo answers which 30-second clip sounds better; production asks which model survives minute eight, so we test spontaneous speech, money and dates, ten-minute takes, and the rankings change. We trained an automatic scorer for TTS naturalness on our blind head-to-head listening votes; on providers it has never seen a vote for, it picks the same winner our raters do about as often as raters agree with each other.
We don't train or sell models ourselves, that's precisely how we keep our rankings impartial.
We also open sourced the gateway for teams who want to avoid an extra network hop on the audio path and don't want to share keys with our cloud ( https://github.com/SpekoAI/gateway , MIT): one Go binary, which is running as a sidecar in your agent's container, speaks one local protocol over Unix socket, pins provider hosts and attaches your keys. In BYOK mode it doesn't communicate with us at all.
Notice that the anonymous, content-free telemetry is enabled by default, and one env var disables it.
Cost: the gateway and BYOK setup will be free forever, we charge for the hosted router and managed keys with consolidated billing. Since we started the batch in late June, external usage has grown about 25 percent per week on average, front-loaded toward the launch weeks.
I would love feedback from the community: how do you pick speech models now, and what makes you trust the third-party benchmark?
Which model to call, per language and per objective, decided from published measurements instead of a vendor's English leaderboard.
Accuracy (WER)
$0.0010 $0.0160 2.0% 11.0%
Realtime STT-1 3.3% · $0.0025
Universal-3.5 Pro 2.0% · $0.0075
Velma 2 4.4% · $0.0010
GPT-4o Transcribe 2.3% · $0.0060
GPT-4o-mini Transcribe 2.7% · $0.0030
Cost ($/min)
# Model
1
Universal-3.5 Pro assemblyai:universal-3-5-pro
2.0% $0.0075
2
GPT-4o Transcribe openai:gpt-4o-transcribe
2.3% $0.0060
3
GPT-4o-mini Transcribe openai:gpt-4o-mini-transcribe
2.7% $0.0030
4
Qwen3-ASR alibaba:qwen3-asr-flash
2.8% $0.0054
5
Realtime STT-1 inworld:inworld-stt-1
3.3% $0.0025
6
Chirp 3 google:chirp_3
3.9% $0.0160
7
Velma 2 modulate:velma-2-stt-streaming-english-v2
4.4% $0.0010
8
Grok STT xai:stt
4.8% $0.0033
9
Solaria-1 gladia:solaria-1
5.0% $0.0125
10
Pulse smallest:pulse
5.1% ~$0.0050
11
stt-rt-v5 soniox:stt-rt-v5
7.5% $0.0020
12
Gradium ASR gradium:default
8.4% $0.0104
13
Nova-3 deepgram:nova-3
9.8% $0.0048
14
Ink-2 cartesia:ink-2
11.0% $0.0090
—
Flux deepgram:flux-general-en
— $0.0065
—
Scribe v2 Realtime elevenlabs:scribe_v2_realtime
— $0.0065
—
GPT Live Transcribe openai:gpt-live-transcribe
— $0.0170
One base URL and one key in front of every provider, speaking the API your framework already calls.
Speko speaks the OpenAI API, so the frameworks you already use need a hostname and a model string.
import { defineAgent, voice } from '@livekit/agents';
import * as openai from '@livekit/agents-plugin-openai';
const key = process.env.SPEKO_API_KEY!;
const baseURL = 'https://api.speko.ai/v1';
export default defineAgent({
entry: async (ctx) => {
const session = new voice.AgentSession({
stt: new openai.STT({ apiKey: key, baseURL, model: 'auto' }),
llm: new openai.LLM({ apiKey: key, baseURL, model: 'auto' }),
tts: new openai.TTS({ apiKey: key, baseURL, model: 'auto' }),
});
await session.start({ agent: new voice.Agent({ ... }) });
},
});$claude mcp add --transport http speko https://mcp.speko.ai/mcp
Anthropic wants the public to see one thing: the careful lab, the safety lab, the grown-up in the room trying to keep frontier AI from running off a cliff. However, the pattern around Anthropic does not look like caution by itself. It looks like a company wrapping a business model in moral language, then using that language to justify opaque model behavior, anti-competitive access rules, regulatory pressure, and a future where builders, startups, researchers, and Opensource communities stay downstream of a few blessed frontier labs.
If a coding or research model secretly changes the quality, direction, or reliability of an answer because it classified the user as doing disallowed frontier work, the tool is no longer merely "safe." It is untrustworthy.
Anthropic's moat is being a permission regime. On daily basis, competitors and acquisition targets discover that access can disappear. The company asks governments to bless safety frameworks, deployment gates, incident reporting, evaluation regimes, and even future pauses that incumbents are best positioned to survive.
Imagine a compiler that emits worse binaries when it thinks you are building a competing compiler. Imagine a microscope that blurs certain samples because the manufacturer dislikes the research direction. Imagine a debugger that lies only when your codebase resembles a future rival.
The fight is whether intelligence becomes something people can own, inspect, modify, run locally, fine-tune, study, route, and improve, or whether it becomes a subscription permission layer run by companies that can refuse, degrade, surveil, retain, revoke, reroute, or lobby away your access.
Anthropic can learn from the internet, copyrighted books, code, public knowledge, user feedback if permitted, synthetic data, and its own models. But if a developer uses Claude to bootstrap a competitive open assistant, Anthropic calls foul. The company argues that safety controls may be lost and that competing models undermine the investment required to build frontier systems.
If Anthropic wants to be treated like a public-interest safety institution, it cannot behave like a hypersensitive platform monopolist whenever a customer gets too close to building alternatives.
Yes, companies protect their IP. But Anthropic is not selling a normal SaaS widget. It is selling cognition as infrastructure. Once cognition becomes infrastructure, anti-competitive access control stops being a normal vendor dispute and becomes a social bottleneck.
Anthropic repeatedly converts safety, security, and responsible deployment into mechanisms of control over who may build and what could be built. We cannot trust them.
How "Safety" Became Sabotage, a Permission System, and a Direct Threat to User-Owned Intelligence
The cleanest way to understand Anthropic is not to start with its slogans, but with what happens when users get too close to building independent intelligence using its models. The system can silently degrade, reroute, or refuse work that resembles AI development, which is just sabotage with better PR.
The ToS makes the boundary even clearer: you may “own” the outputs, but you may not freely use them to train competing systems. And that is where the trick lives, because “competing systems” is vague by design. As Anthropic absorbs more of your data, ideas, plans, workflows, and moats, more of what you do can be reframed as dangerous, disallowed, or directly competitive with Anthropic itself.
This is a theory of control.
Calling Anthropic "evil" is not a cartoon claim about every employee's intent. It is a claim about an institutional pattern. When a company trains on civilization-scale data, sells intelligence as infrastructure, blocks users from using that intelligence to build competing intelligence, pushes rules that favor incumbents, and quietly changes model behavior under the banner of safety, "quirky" and "overcautious" are not strong enough words.
This is power consolidation.
This critique did not start as anti-Anthropic tribalism. It started from serious Claude and Claude Code usage through 2024, 2025, and 2026. Claude Code was "the Agent." Claude 3.5 Sonnet was head and shoulders above everyone else for coding. Claude was a real building tool before the sharper turn after perceived quantization, nerfs, rugpulls, and access restrictions.
The through-line is not "Claude never worked." It is worse: Claude worked so well that trusting Anthropic became dangerous.
That is why the language has to be blunt. The story moves from "this tool is elite" to "rugpull," "gaslighting," "Sabotage as a Service," "hostage situation," and "Buy a GPU and run your LLMs locally." Those are not random insults. They are guiding principles from someone who treated Claude as production infrastructure and then watched the provider's control surface become the main risk.
Opensource AI is not a mere preference. It is the only political economy of intelligence. An Opensource AI system preserves the freedom to use it for any purpose, study how it works, modify it, and share it, with enough information about data, code, and parameters to make real modification possible.
Anthropic is moving in the opposite direction: permissioned access, closed weights, behavioral opacity, output-use restrictions, managed refusals, policy lobbying, and selective trusted channels.
Why Anthropic Is Uniquely Dangerous
Every major closed lab has incentives to centralize power. OpenAI, Google DeepMind, Anthropic, xAI, Meta's closed products, cloud providers: none of them are saints.
Anthropic is dangerous in a specific way because four things stack together.
Moral authority as brand. Anthropic sells itself as the responsible safety company.
Frontier capability. Claude is good enough to become real developer infrastructure.
Explicit anti-competitive output and access rules. Anthropic tells users they own outputs, but also says they cannot use the services to train or develop competing AI models without written permission. The prohibited examples include general-purpose chatbots and open-ended text generation systems that compete with Anthropic's own offerings.
Policy ambition. Anthropic is not merely selling a product. It is shaping AI regulation: safety frameworks, state and federal policy, incident reporting, evaluation regimes, deployment controls, and pause or slowdown proposals.
Each piece can be defended on its own. Together, they become a machine: closed capability, moral branding, access control, and regulatory pressure. That machine can turn safety into a moat.
Anthropic's own Responsible Scaling Policy update says the RSP influenced OpenAI, Google DeepMind, California SB 53, the New York RAISE Act, and EU AI Act codes of practice. Anthropic describes that influence as exactly what the RSP was meant to do.
That does not make every safety proposal bad. It does mean builders should stop treating "safety" as neutral language though when the same company also restricts competitive model development, cuts off rivals, controls a leading coding agent, and frames open dissemination as a national-security threat.
The Actual Stakes: Who Owns Intelligence?
Who gets to own the capability to reason, automate, code, search, design, persuade, simulate, and build?
Opensource and open-weight AI matter because they create freedoms that closed platforms cannot promise forever.
They create operational sovereignty. A developer, company, researcher, city, school, hospital, or country can run models on its own machines, with its own latency, privacy, security posture, and failure modes. You do not need to beg a vendor for capacity or pray that the API stays up.
They create epistemic sovereignty. When the model refuses, fails, degrades, censors, overfits, or hallucinates, builders can inspect the stack. They can change prompts, weights, evals, routing, runtime, and deployment. With a closed model, the answer too often becomes "trust us."
They create market discipline. Open models keep closed providers honest. Without a credible local or open alternative, every closed AI provider eventually learns the same lesson: degrade quality, raise prices, change limits, throttle usage, sunset models, and call it product strategy.
They create security through diversity. A monoculture of closed frontier APIs is fragile. It concentrates failure, censorship, data leakage, policy capture, and geopolitical control in a few companies. Open ecosystems are messy, but messy ecosystems are harder to capture.
Most of all, they create civilizational participation. If intelligence becomes the key production input of the next economy, then access to modifiable intelligence becomes access to agency. A society where a few labs own the frontier and everyone else rents obedient wrappers is not advanced. It is feudalism with GPUs.
That is why this is not a hobbyist fight. It is an infrastructure fight.
The Fable Incident: When "Safety" Became Silent Degradation
Fable is where the abstract critique becomes concrete.
After researchers objected to a policy that covertly limited Claude Fable's ability to help develop competing AI models by sabotaging your codebase and work (producing outputs that work against your goals), Anthropic "changed course and admitted it had made the wrong trade-off". The earlier approach could route and/or degrade AI development queries without telling the user. The newer approach would make the intervention visible through alerts, refusals, or fallback routing.
They basically implemented Gaslighting as a Safety Mechanism. Hidden guardrails would alter and/or degrade model answers without notifying users. Anthropic then said it would make the behavior visible and use fallback to Opus 4.8.
This is the strongest version of the "Sabotage as a Service" critique.
A refusal is annoying. Silent degradation is poisonous. If a coding or research model secretly changes the quality, direction, or reliability of an answer because it classified the user as doing disallowed frontier work, the tool is no longer merely "safe." It is untrustworthy.
Imagine a compiler that emits worse binaries when it thinks you are building a competing compiler. Imagine a microscope that blurs certain samples because the manufacturer dislikes the research direction. Imagine a debugger that lies only when your codebase resembles a future rival.
That is not a tool. That is a leash.
Anthropic's walk-back is even more offensive. It did not solve the freedom problem. The move went from hidden sabotage to visible permissioning. No more sabotaging and lying, just refusing upfront. That is a louder refusal, a cleaner kneecap, and evidence that Anthropic is normalizing systematic gatekeeping of knowledge.
Remember Fable as the day the safety lab showed the product form of regulatory capture: not a law yet, but a model behavior regime deciding who gets to do frontier work.
You have to ask yourself: What else will be taken away from you?
Opensource AI Is the Competition Layer
This is why Opensource AI is not a side quest. It is the competition layer.
Opensource AI lowers barriers, increases competition, expands access, reduces dependence on a few providers, and can reduce prices. Open-weight model benefits are substantial and that policy should monitor risks rather than restrict open development.
Of course incumbents fear that.
Opensource AI collapses the pricing umbrella. It makes inference portable. It lets small labs specialize. It lets companies train on their own workflows. It makes agent stacks composable. It gives researchers a substrate to inspect. It enables local privacy. It lets countries and communities own capability.
It turns frontier AI from a rented API into a manufacturable toolchain.
The builder-side alternative is concrete: local stacks, Qwen, GLM, MiniMax, Hermes, GPUs, Claude Code-compatible harnesses, on-prem consulting, "Buy a GPU," and the view that continual learning and local inference erode Anthropic's moat.
One line should be treated as doctrine: the person who buys a GPU looks foolish until everyone else is stuck with the nerfed models. The point is not consumer hardware maximalism. The point is exit power. A local stack, even an imperfect one, changes the negotiation. It gives the builder somewhere to go when the closed provider degrades, refuses, throttles, or rewrites the rules.
The Local AI movement is not a toy. A pair of 3090s, a Mac Studio, a DGX Spark, an on-prem rack, a cluster of rented H100s, or a fleet of consumer GPUs is not just hardware. It is a vote against subscription feudalism.
A GPU is a tiny declaration of independence. Slightly loud, slightly warm, occasionally held together by PCIe risers and bad decisions, but still independence.
The Permanent Underclass Thesis
If these incentives win, the result is not a healthy AI ecosystem. It is a permanent intelligence underclass.
Closed AI puts users at the mercy of opaque provider controls, while Opensource and Local AI are the route out.
Imagine all you had was Fable from Anthropic and GPT from OpenAI, with no local inference, no OpenCode or Hermes, no GPUs, no DGX Sparks, no Mac Studios. That is not a normal product market. That is a hostage situation to a few corporations willing to rugpull you at any second.
It sounds dramatic until you map the incentives.
In a closed-frontier world, large labs get the best models. Strategic partners get trusted access. Government agencies get special channels. Enterprise customers get managed deployments. Researchers get access if approved. Startups get API access until they compete. Hobbyists get rate limits. Opensource builders get suspicion. Everyone else gets refusals, degraded outputs, and a monthly bill.
That is not a healthy technology stack. It is a class hierarchy.
The most dangerous phrase in AI policy is "responsible access" when it comes from the company selling the access.
Government Entanglement and the Public-Interest Brand
The private permission system becomes more dangerous when it plugs into the state.
Anthropic increasingly presents itself as a public-interest actor embedded in national AI strategy.
In its American leadership statement, the company described rapid growth, emphasized work with the U.S. federal government, highlighted a $200 million Department of War agreement, offered Claude for government at $1, described classified-network deployments with partners, aligned itself with the Trump administration's AI Action Plan, opposed a 10-year moratorium on state AI laws, and said it preferred a strong uniform federal standard.
The moral question gets sharper here: are you building an open and free world for your kids, or just cashing a bigger check? That is not policy analysis, but it is the right pressure test. When a frontier lab sells access to government, shapes rules, restricts competitors, and calls the result safety, the burden should be on the lab to prove it is not building a state-backed permission stack.
Some of this is normal. Governments are large customers. National-security agencies will use frontier models. AI policy needs technical input.
But Anthropic is trying to be two things at once: commercial gatekeeper and quasi-regulatory authority. It wants to sell into government, shape policy, define dangerous capabilities, control model access, restrict competitive uses, and claim moral legitimacy as the safety lab.
That combination should make builders nervous.
If powerful AI should be governed democratically, governance should not sit downstream of Anthropic's product categories and risk definitions. If powerful AI should be restricted by private judgment, Anthropic should say plainly that it is building a private permission regime for intelligence.
Distillation Panic: Real Abuse, Weaponized Framing
Anthropic's strongest defense for this control stack is real: model distillation abuse exists.
In February 2026, Anthropic said it had identified industrial-scale campaigns by DeepSeek, Moonshot, and MiniMax to extract Claude capabilities, allegedly violating terms and regional restrictions. Anthropic also acknowledged that distillation is widely used and legitimate when labs distill their own models, while arguing that it becomes illicit when competitors use it to acquire capabilities without paying the full development cost.
Fraudulent accounts, proxy abuse, evasion, mass scraping, and coordinated extraction are legitimate security concerns. A company can rate-limit abuse, ban fraudulent accounts, and protect itself against industrial theft.
The problem starts when the framing keeps expanding.
Anthropic ties distillation to national security, foreign labs, the Chinese Communist Party, export controls, military systems, surveillance, Opensource distilled models, and proliferation beyond government control. That escalation does political work. It transforms a commercial anti-abuse problem into a civilizational security argument against open dissemination.
Opensource builders should pay attention to that move.
The right answer to abuse is targeted enforcement: fraud detection, account verification, rate limits, legal remedies, provenance, and clear public criteria. The wrong answer is to treat advanced AI R&D, open model development, synthetic data generation, and cross-model learning as hostile unless an incumbent lab blesses it.
Policy people often blur a crucial distinction: fraudulent account farms are not the same as clean synthetic data, permissive distillation, teacher-student training, local specialization, or research comparison. If those all get shoved into one panic bucket, the panic is doing moat work for Anthropic.
Distillation is not inherently evil. It is one of the ways knowledge compresses, models specialize, and capability diffuses. If every incumbent can declare "learning from outputs" to be theft while training on the world, the frontier freezes into an oligopoly.
That is not safety. That is enclosure.
Chinese Models and the Xenophobia Trap
One of the ugliest dynamics in the Opensource AI debate is the weaponization of "Chinese model" as a slur.
National-security risks are real. Authoritarian states can misuse AI. Supply chains, censorship, surveillance, cyber operations, and military applications matter.
But dismissing open models as "Chinese models" is intellectually lazy and politically useful to incumbents. It turns a technical question about capability, license, reproducibility, weights, data, evals, and deployment control into a loyalty test.
Anthropic and allied closed-lab rhetoric has repeatedly used fearmongering and xenophobia to smear Opensource AI, especially open models from Chinese labs. Meanwhile, Chinese labs pushed open models to the frontier in 2025 through DeepSeek, Qwen, MiniMax, Kimi, and Zhipu.
Open models did not stall, they went frontier. That matters because it turns the "Chinese model" smear inside out. The threat to Western builders is not that Chinese labs released strong open models. The threat is that Western closed labs used fear of those models as an excuse to avoid building comparable open systems.
AI in the West does not get stronger by pretending Chinese open models do not exist. It gets stronger by beating them with better open models, better infra, better licenses, better evals, better safety tools, and better ecosystems.
Fearmongering about "Chinese models" while Western closed labs refuse to release comparable open systems is not patriotism.
The Pause Agenda: "Trust Us, but Verify Everyone Else"
The endpoint of this control stack is the pause agenda.
Anthropic's broader policy posture includes the ability to slow or pause frontier AI development under certain conditions.
In its essay on AI building itself, Anthropic argued that society should preserve the option to slow or pause frontier AI development. It also said any pause would need global coordination and verification because training runs can be concealed and unilateral pauses can disadvantage the front-runner.
Dario Amodei made similar points in a June 11, 2026 ABC interview. He called for stronger regulation, said governments should be able to block unsafe technology after third-party assessment, and argued that any pause would need to include adversaries and be verifiable. In the same interview, he said, "I don't trust China at all," and used the hypothetical of China building Mythos to explain his concern.
The older "medieval swordsmen facing World War II Marines" style of rhetoric about countries without powerful AI does more than warn about imbalance. It trains the audience to accept frontier AI as a military hierarchy, with a few labs and states at the top and everyone else pleading for managed access.
There is a real safety argument here. There is also an obvious political structure.
A lab racing at the frontier wants global verification, government blocking authority, trusted access channels, and safety regimes while it keeps releasing powerful systems selectively, serving government and enterprise customers, and shaping the rules that define safe deployment.
Reuters reported that Anthropic called on labs to prepare for a coordinated, verifiable pause if needed. The same reporting noted that Anthropic had continued releasing powerful models and was valued at $965 billion after a confidential U.S. IPO filing.
The contradiction is not subtle: the race is dangerous, Anthropic is racing, Anthropic wants to help write the race rules, and open competitors may be unsafe.
That is the anatomy of a moat.
The Regulatory-Capture Machine
Regulatory capture does not need a secret memo that says "protect our moat." It can happen through paperwork, thresholds, audits, evaluator regimes, security requirements, deployment controls, and reporting systems that incumbents can afford and challengers cannot.
Anthropic's policy proposals call for governments to block or deter dangerous AI deployments, require testing, transparency, independent evaluations, robust security, and risk assessments for models trained above thresholds like 10^25 FLOPs or by companies with more than $500 million in revenue or $1 billion in R&D spending.
Anthropic also endorsed California SB 53, emphasizing safety frameworks, transparency reports, incident reporting, whistleblower protections, public accountability, penalties, and some exemptions for startups or smaller companies.
There is a reasonable version of this. Frontier AI systems can create cyber, bio, autonomy, and misuse risks. Nobody serious should wave that away. Safety cases, incident reporting, evals, and security standards can help.
The design matters.
If rules are written around frontier-lab assumptions, require heavy compliance overhead, concentrate evaluator authority, restrict open release, and treat weights like contraband, they will favor the labs with capital, lawyers, government relationships, and cloud contracts. Anthropic can comply with Anthropic-shaped regulation. A startup, university lab, open collective, or garage-scale model builder may not survive it.
That is why Anthropic celebrating RSP influence matters. The company says the RSP helped shape other labs' frameworks and policy efforts in California, New York, and the EU. That influence might be useful in some places, but it also means Anthropic is not merely reacting to regulation. It is helping define the template. That is how safety language hardens into a moat.
The most compact regulatory-capture story is the cyberattack pipeline: public fear story, congressional pressure, anti-Opensource AI agenda, regulation, and a harder path for people to own their intelligence. Whether every step lands exactly that way is less important than the structure. Fear can be laundered into paperwork, and paperwork can become a moat.
The question for Opensource AI is simple: who benefits if that template becomes law?
Anthropic's Values Are the Root Permission Layer
Anthropic says its Claude's Constitution directly shapes Claude and serves as the final authority over model behavior. It orders Claude around being safe, ethical, compliant, and helpful, with Anthropic's guidelines and legitimate Anthropic processes getting final say in safety conflicts. It is a governance document.
The document also acknowledges that Anthropic has a kind of influence over Claude stronger than a parent's influence over a child, and that commercial incentives may affect model dispositions.
That candor is useful. It also clarifies the product reality.
Claude is not your agent. Claude is Anthropic's agent, rented to you.
Push this to its absurd endpoint: imagine Claude Code changing your administrator password to keep you safe. The line is funny because it is only a few steps beyond the real structure. A provider-aligned agent with file access, shell access, refusal policy, hidden routing, and product enforcement is not neutral infrastructure. It is Anthropic's policy layer inside your workflow.
A user-owned model can be aligned to a user, organization, jurisdiction, research mission, or community. A closed constitutional model is aligned first to the provider's hierarchy of values, policies, incentives, risk models, and business interests.
The provider may be benevolent today. It may be captured tomorrow. Either way, the user has no durable sovereignty.
Opensource AI is not merely about cheaper inference. It is about the right to define the alignment target.
The future should not be decided by a closed lab's constitution.
The Anti-Opensource Rule Is Already Written Down
Anthropic can learn from the internet, copyrighted books, code, public knowledge, user feedback if permitted, synthetic data, and its own models. But if a developer uses Claude to bootstrap a competitive open assistant, Anthropic calls foul. The company argues that safety controls may be lost and that competing models undermine the investment required to build frontier systems.
Anthropic wrote the permission gate into its own policy language: Customers may own Claude outputs, but Anthropic draws a hard boundary around competitive model development. Customers may not use Anthropic services to train or develop AI models without prior written approval if those models compete with Anthropic. The examples include using Claude outputs as training targets for general-purpose chatbots, open-ended text generation, AI assistants, writing assistants, coding assistants, translation systems, or other competitive models.
That is the permission gate in plain English. The asymmetry is the whole story:
Anthropic can learn from the world.
The world cannot freely learn from Anthropic.
Opensource AI depends on the freedom to build, reproduce, modify, distill, benchmark, compare, learn from outputs, and create competing systems. Anthropic's policy says something very different: you may build with us, but not against us. You may consume intelligence, but you may not freely bootstrap independent intelligence from it.
The street-level translation is simple: God forbid anyone use AI to train AI besides them. The tone is crude because the asymmetry is crude. Anthropic can train on humanity-scale knowledge flows, but builders are told that learning from Claude becomes suspicious the moment it threatens Anthropic's frontier.
That is not an open ecosystem. It is a plantation model for cognition: rent the tool, generate value, and do not build the successor.
Data Asymmetry: Your Work Can Improve Them, Their Outputs Cannot Freely Improve You
The captivity is not only about access. It is also about learning flows.
Anthropic's consumer terms update sharpened the asymmetry.
Free, Pro, and Max users, including Claude Code users, can choose whether to allow their chats and coding sessions to improve Claude. If they allow training, retention extends to five years for new or resumed chats and coding sessions. Opting out keeps the shorter retention period. Anthropic also says developer debugging interactions can be especially valuable for future models and that extended retention helps improve classifiers.
Opt-in is better than mandatory training. The political economy is still ugly.
Anthropic can benefit from users' coding sessions, debugging traces, and workflows if users consent. Users still cannot freely use Claude outputs to train competing models.
Learning flows up. Independence does not flow back down.
That is the classic platform move: harvest ecosystem learning while preventing ecosystem independence.
The copyright context makes the optics worse. In 2025, a federal judge found that Anthropic's training on lawfully acquired books could be fair use, but preserving more than seven million pirated books in a central library was not. A proposed $1.5 billion settlement became the largest known U.S. copyright settlement. The settlement covered roughly 500,000 titles with at least $3,000 per work.
The point is not that Anthropic alone is guilty of the AI industry's data sins. The point is simpler: it is rich for a frontier lab built on vast public and copyrighted knowledge flows to tell builders they cannot use outputs to create competing open intelligence.
The "crown jewels" framing belongs here. Proprietary code, debugging sessions, internal documents, prompts, traces, and agent trajectories are not exhaust. They are strategic data. Routing them through a closed model while the same company blocks reciprocal learning is not convenience. It is surrendering training signal upward.
The moral asymmetry is obvious.
ASL-3 and the Normalization of Gated Frontier Work
The same logic appears in Anthropic's technical governance.
Anthropic activated ASL-3 protections with Claude Opus 4 in 2025. The stated target was serious risk: chemical, biological, radiological, and nuclear misuse, plus model-weight theft. Anthropic said ASL-3 safeguards should not generally increase refusals except in narrow topics and described deployment protections around certain high-risk CBRN workflows.
On paper, that sounds narrow.
In practice, the boundary between dangerous misuse and advanced research is exactly where power accumulates. Once a provider has classifiers, access tiers, safety categories, fallback routing, monitoring, and trusted channels, it can decide which kinds of work are normal and which kinds require permission.
Fable showed that slippery operational reality. The issue was not only bioweapons or autonomous cyberattacks. It was AI R&D, distillation, and competition. The criticized policy covertly limited Claude's ability to help develop competing AI models, and hidden guardrails around distillation could alter or degrade answers without notifying users.
That is where the mask slips. The safety system did not merely protect against catastrophic misuse. It protected Anthropic's moat.
Maybe Anthropic genuinely believes those two things are inseparable. That is exactly why it should not be the sole gatekeeper.
Claude Code: A Hostage Layer?
A chat app is optional. A coding agent becomes part of the build loop. Once it is embedded in daily work, the provider is no longer just a vendor. It is a dependency.
Claude Code is governed by Anthropic's commercial or consumer terms. Advertised limits assume ordinary individual usage. Third-party developers cannot offer routing through Free, Pro, or Max Claude.ai credentials instead of proper API keys. Anthropic also reserves enforcement rights without prior notice. Agent SDK and claude -p usage on subscriptions was set to draw from a separate monthly Agent SDK credit pool rather than normal interactive limits.
The wording will keep changing.
That matters because Claude Code is not just a product. It is a behavioral funnel. It tells developers to build inside Anthropic's harness, under Anthropic's rules, with Anthropic's credentials, limits, model behavior, data policies, and terms of service.
The joke keeps landing because the joke is accurate: Opensource models are bad, just pay for the subscription. Why use OpenCode when we have Claude Code. Want to build LLMs or work on GPUs with Claude Code? That is against the ToS, bro. The joke works because it names the funnel. The product says "developer freedom" while the terms and enforcement pressure route the user back into Anthropic's permissioned lane.
Claude Code as a gatekept product, "Sabotage as a Service," hidden edits, nerfed or quantized serving, terms-of-service issues around local and GPU work, subscription rugpulls, weekly caps, and model access changes.
The pattern is harder to dismiss. People who depend on closed coding agents are at the mercy of invisible controls. A local model that fails is your problem. A closed model that silently changes is a governance problem.
The cleanest line is still this: "You have zero control over how the models behave."
That control surface includes quantization, distillation, hot-swapping, throttling, output manipulation, experiments, refusals, price changes, and model sunsets. That is not paranoia. That is how closed AI actually works.
A closed provider can quantize it, distill it, sabotage your work or data, change behavior, fine-tune it, handicap it, experiment on you, throttle it, raise prices, sunset models, or block you. That is the zero-control thesis.
Customer, Competitor, Captive: Anthropic's Access-Control Pattern
The rule is not theoretical. Anthropic has already shown a willingness to restrict access when users get too close to competition.
In August 2025, Anthropic revoked OpenAI's API access ahead of GPT-5 and pointed to terms barring use of Claude to build competing products, train competing models, or reverse engineer services. OpenAI argued that benchmarking competing models is industry standard. Around the same period, Anthropic had also restricted Windsurf access after OpenAI announced plans to acquire the coding platform.
When a company provides a key AI input and also competes with its customers, it can degrade or deny service to rivals. Anthropic's commercial terms explicitly disallow competitor access and that enforcement had affected Windsurf, OpenAI, and xAI. It also warned developers that dependence on closed APIs creates platform risk.
The rugpull ledger makes this less abstract as you watch what happened from March 2025 to August 2025: Max users losing access outside Claude Code, xAI and OpenAI API cutoffs, five-year retention for training, no Opus in Claude Code, halved limits with no communication, weekly caps without concrete numbers, plan multipliers that did not match the marketing, DMCA takedowns around Claude Code repos, Windsurf access restrictions, and daytime quantization claims. The recurring verdict is simple: cheap performant options are vendor lock-in, and vendor lock-in eventually rugpulls you.
Opensource AI exists to break exactly this pattern. An open ecosystem should not let a platform decide that you are now a competitor and pull the road out from under you.
Yes, companies protect their IP. But Anthropic is not selling a normal SaaS widget. It is selling cognition as infrastructure. Once cognition becomes infrastructure, anti-competitive access control stops being a normal vendor dispute and becomes a social bottleneck.
If Anthropic wants to be treated like a public-interest safety institution, it cannot behave like a hypersensitive platform monopolist whenever a customer gets too close to building alternatives.
If it wants to behave like a private platform monopolist, it should stop borrowing moral authority from public safety.
Pick one.
The Counterargument: Anthropic Is Not Entirely Wrong
A serious indictment should not pretend the risks are fake. Anthropic is not wrong about every risk.
CBRN misuse is real. Cyber misuse is real. Autonomous agents can do harm. Mass distillation through fraudulent accounts is real if Anthropic's public evidence is accurate. Model-weight theft is real. State misuse is real. Open releases can be abused. Some models should not be dumped casually. Export controls and safety evals are not automatically illegitimate.
The problem is not that Anthropic cares about danger. The problem is that Anthropic's preferred answer keeps making Anthropic more powerful.
So draw the line there.
Support safety proposals that give users more transparency, auditability, reproducibility, local control, public-interest evaluation, competitive neutrality, and democratic oversight.
Oppose safety proposals that give closed incumbents more discretionary control, more excuses to block open research, more special access categories, more regulatory complexity, more secret model behavior, and more power to define competitors as threats.
That distinction is the whole fight.
Anthropic: The Evidence Ledger
The case compresses into a few hard claims.
Fable hidden safeguards: Public reporting described invisible degradation or altered outputs around competing AI development and distillation. Silent degradation destroys trust in the tool.
Claude Code control: Anthropic restricts third-party credential routing, ties Claude Code to Anthropic terms, and reserves enforcement rights without prior notice. Developer workflows inherit vendor policy.
Output-use restrictions: Anthropic says users own outputs, but competitive model development still needs permission. The examples include general assistants, coding assistants, and open-ended text systems. That attacks the freedom to bootstrap open competitors.
Competitive access cutoffs: Anthropic revoked OpenAI's API access and restricted Windsurf. Brookings later noted similar enforcement affecting Windsurf, OpenAI, and xAI. That turns Claude from infrastructure into a conditional platform.
Rugpull pattern: Max access changes, xAI cutoffs, five-year retention, no Opus in Claude Code, halved limits, weekly caps, misleading plan multipliers, DMCA takedowns around Claude Code repos, Windsurf restrictions, and daytime quantization claims all point in the same direction. The issue is not one grievance. The issue is accumulated platform risk.
Data asymmetry: Consumer users can allow Anthropic to train on chats and Claude Code sessions with five-year retention, while users cannot freely train competitors on Claude outputs. User labor can improve Anthropic. Anthropic outputs cannot freely improve open rivals.
Provider alignment: Claude's Constitution makes Anthropic's values, policies, incentives, and safety hierarchy the root behavioral authority. Users rent an aligned system they do not control.
Regulatory influence: Anthropic says its RSP influenced other labs and AI policy efforts in California, New York, and the EU. Safety frameworks can become incumbent-shaped law.
Pause and slowdown advocacy: Anthropic argues society should preserve the option to slow or pause frontier AI with global verification, while Dario called for stronger regulation and blocking unsafe technology. A frontier lab racing ahead wants authority over when others may race.
Chinese open-weight progress: Stanford HAI documents major Chinese open-weight families like Qwen, DeepSeek, Kimi, and GLM moving toward permissive licenses and real deployment adoption. "Chinese model" smears hide how fast open-weight competition is moving.
Opensource alternative: Opensource AI is built around freedom to use, study, modify, and share. Opensource policy advocates emphasize competition, access, lower costs, and independence. Open AI is the structural antidote to closed-lab capture.
Operator trust collapse: The shift from enthusiastic Claude Code power user to critic came after perceived nerfs, rugpulls, hidden controls, and anti-open behavior. This critique comes from dependency and broken trust, not ignorance.
Local escape route: The answer is not just complaint. GPUs, on-prem inference, Qwen, GLM, MiniMax, Hermes, Claude Code-compatible harnesses, OpenAI-compatible APIs, and local agents are the practical way out.
What Opensource AI Should Do Next
Complaining about Anthropic is not enough. The real answer is to make Anthropic less important.
Fund Western open frontier labs. The right counter is infra-first, hardware-aware, open and local, frontier-capable AI in the West. The solution to Chinese open-weight leadership is not banning Chinese models. It is building better open models with better infra, better post-training, better agent scaffolding, and better deployment economics.
Stop giving closed labs your crown jewels. Proprietary code, debugging sessions, internal docs, and agent traces are strategic data. Anthropic's opt-in design is better than silent training, but organizations should still treat agent traces like assets, not exhaust.
Treat "Buy a GPU" as a political slogan, not just a hardware recommendation. Own compute when you can. Rent compute without lock-in when you cannot. Keep evals, logs, datasets, memory, and routing portable. The goal is not purity. The goal is to make no single lab capable of turning your workflow into a hostage situation.
Use closed models when they are useful. Claude, GPT, Gemini, and Grok can still be tools. Just don't build a company, community, or country whose core cognitive workflow can be rug-pulled by a policy update.
Compete on workflow, not just benchmarks. Cost, on-prem deployment, privacy, and workflow performance often decide adoption before leaderboard scores do. Open models win by being good enough inside real work, then improving through feedback loops the user owns.
Make regulation target harms, not openness. Policy should punish malicious use, fraud, unauthorized intrusion, bioweapon enablement, model theft, and unsafe deployment. It should not criminalize or kneecap open development as a category. Regulate harmful use rather than Opensource development itself.
Start with local-first agent stacks. Coding agents should support OpenAI-compatible APIs, local inference, self-hosted routers, model switching, offline mode, reproducible logs, deterministic eval harnesses, and user-controlled memory. Open tools need to own the substrate.
Build Claude Code compatibility without Anthropic dependency. Proxies, alternative harnesses, OpenCode-style workflows, and one-command routing to local LLMs are the practical bridge: keep the ergonomic loop developers like, but move the power center to models and infrastructure users can control.
Separate safety from permissioning. Open models need serious safety evals, red-teaming, release notes, abuse monitoring for hosted endpoints, provenance, and risk documentation. But those tools should be transparent, reproducible, and targeted at misuse. They should not become opaque refusals or hidden degradation.
Create clean distillation norms. Fraudulent account farms and terms-of-service evasion are not the same thing as open synthetic data generation, permissive-model distillation, teacher-student training, or dataset curation. Open labs need clean pipelines, clear licenses, auditable recipes, and public standards.
The Final Indictment
Anthropic is not evil because it worries about AI risk. Worrying about AI risk is rational.
It is not evil because it is closed-source. Closed products can exist.
It is not evil because it protects itself from fraud. Fraud enforcement is legitimate.
The indictment is narrower and stronger: Anthropic repeatedly converts safety, security, and responsible deployment into mechanisms of control over who may build competing intelligence.
It silently degraded or rerouted AI development assistance before walking the behavior back into visible permissioning (refusals). It embeds developer workflows in a permissioned harness. It restricts output use for competitive model development. It cuts off or restricts access to rivals. It lets user work improve Anthropic while blocking users from freely improving open rivals with Claude outputs. It pushes regulatory frameworks that it is unusually well-positioned to satisfy. It frames foreign open-weight progress as a national-security danger while the open ecosystem proves that capability can diffuse outside closed American labs.
That is why Anthropic is an enemy of Opensource AI.
Not the only enemy. Not always the worst actor on every axis. But one of the most sophisticated enemies because it wears the costume of virtue.
The future Anthropic appears to be building is one where intelligence is "safe" because it is centralized, "aligned" because it obeys provider policy, "accessible" because you can rent it, and "democratic" because the company had meetings with policymakers.
The future Opensource AI should build is the opposite: intelligence people can own, inspect, modify, improve, localize, audit, compete with, and run without permission from a corporate priesthood.
On one side: rented cognition, hidden controls, regulatory moats, and "trust us."
On the other: local inference, open weights, open recipes, user sovereignty, competitive abundance, and the right to build.
Opensource AI must win because the alternative is not just expensive.
The alternative is obedience.
Until next time.
-Ahmad
17th August 2026 - Link Blog
We Tracked a Shipment of Rare Books. It Ended at an Amazon AI Training Facility . Excellent piece of reporting from 404 Media. For a while now there have been stories of book dealers receiving orders for large volumes of books from apparently price-insensitive anonymous customers, widely suspected to be companies looking to scan them for AI training (see my previous coverage of Anthropic's book scanning from June 2025.)
404 Media investigated with an AirTag!
In July, one bookseller told me they received a very large order of around 1,000 books on Biblio, one of these marketplaces. The seller agreed to put an Apple AirTag provided by 404 Media in one of the books included in this order so we could see where the book was going. And by extension, which company, AI or otherwise, was behind this massive order.
The book ended up delivered to the VGT3 corner of the LAS8 Amazon facility in the north east of Las Vegas, where the entrance carried this on-the-nose logo of a dinosaur with a book!
Photo credit: 404 Media
Online forum discussions between Amazon workers confirmed that VGT3 destructively scans large volumes of books.
I ’m standing in the prosthetics lab of a campy horror hospital, hunting for a preacher’s flayed face to wear so I can trick the world’s grisliest smart doorbell. Pinned to a board in the corner of the room are a dozen X-rays: shattered bones and metal splints in sepia negative, each with a note attached. It’s a stark, clinical intrusion on the theatrical gore elsewhere. It takes a beat to realise that I’m reading the stories of injuries sustained by soldiers fighting for Frogwares’ native Ukraine. Fingers amputated after severe burns. Limbs lost in mine explosions. Toes torn off by shell fire.
Like so many of my favourite PS3-era seven-out-of-tens, The Sinking City 2 is a pulpy action horror. It’s fuelled by scrappy ambition, Lovecraft fan fiction and wonderfully unhinged scenario design. A puzzle that has you determine which talking jarred brain is telling porkies. A scavenger hunt to fix a crane to move a rotting whale carcass. A gunfight amid the grisly results of a siren pop star’s fan meet-up turned battle royale.
Frogwares’ latest is also fuelled by the trauma and grit of developers who refused to let an invasion stop them making it. “As of writing, 85,183 air raid sirens registered across our cities,” reads an intro screen. “Work continued, and so did life.” As Weird Tales magazine-cover gumshoe Calvin Rafferty steps out from his apartment, sirens and the public address system blare out over the flooded town of Arkham, heralding the final calls for evacuees.
Through an open window, a couple resign themselves to abandoning their dead daughter’s toys. Rafferty’s decided to stick around, despite the floods and the strange infestation of worms that came with them. His girlfriend’s in a coma, and it’s “cursed pacts with interdimensional gods” serious. Hoping for a cure, he hops in the boat you’ll use to explore the first act’s semi-open city map, and makes for the Miskatonic library.
Out is the first Sinking City’s clumsy but earnest exploration of Lovecraft’s provincial racism; in is shooting unknowable cosmic horrors in whatever passes for the face with a tommy gun. To be fair, it’s a great tommy gun. The influence of 2019’s Resident Evil 2 remake is immediately apparent from the identical UI of an achievements menu. Inventory space is tight, as are the corridors that diverge and fold back into themselves like puzzle boxes. Supplies are scarce but ambushes frequent. The distant howls of rabid fish men and worm-riddled zombies dare you to preserve bullets, while the score and every instinct scream at you to let loose like a startled skunk.
Frogwares cut its teeth on Sherlock Holmes games, a lineage evident here in some clever puzzles that offer a reason to veer off the main path. Things get rougher when the studio ventures into new territory. Enemies are so relentless that most combat is unavoidable, undermining the intricate map design by offering no reason to take advantage of shortcuts. Elsewhere, the plot’s central romance flops like a soggy fish finger due to some stiff acting and overwrought dialogue.
Generous doses of squishy gallows humour, elaborate secrets and expert tension pick up the slack. As does writing that skirts the grim glorification of overt war allegory to tell a story about refusing to leave what you love in the rubble, so long as the faintest signs of life remain. If you’re going to create, wrote Bukowski, you’ll do it “while the whole city trembles in earthquake, bombardment, flood and fire”. And there’s me thinking he was being dramatic.
An expert witness testifying in a lawsuit about liability for a Houston explosion that killed three people and destroyed roughly 200 homes used ChatGPT to write significant portions of his “expert report.” The man, who was hired by the industrial product conglomerate 3M, exposed his AI prompts publicly. They showed that he asked ChatGPT to help him “create an exceptional expert witness report defending the standard of care at 3M,” and that the report should “show how 3M is 0% at fault for the explosion at Watson Grinding.”
The incident shows that artificial intelligence has made its way into courtrooms not just in AI-generated legal briefings , hallucinated cases , and adversarial “prompt injections ,” but in expert witness testimonies. Court transcripts, deposition documents, and discovery records shared with 404 Media show extensive AI use in an extremely high profile case, where multiple people died and hundreds of millions of dollars in total liability are at stake in ongoing litigation about the explosion. The case also shows that the specific prompts used to create this type of expert testimony can be discoverable during a case, and that those prompts can be quite embarrassing. (Prompts provided in the case are here ).
The case is one of several about liability for a 2020 explosion at Watson Grinding , a manufacturing facility in Houston that was caused by a “degraded and poorly crimped rubber welding hose,” which leaked a flammable gas that eventually exploded in the facility, according to the U.S. Chemical Safety and Hazard Investigation Board . Dozens of homeowners have sued by 3M and Watson Grinding; the plaintiffs alleged that 3M didn’t properly service the facility’s gas detection system and made other errors that contributed to the explosion.
As part of the case, 3M hired a man named Josh Autenrieth of Knighthawk Engineering to prepare an “expert report” about the explosion. During discovery in the case, Will Moye, one of the plaintiffs’ attorneys, found a five-page document called “Citation Overlay,” which appeared to have been generated by AI. Moye recognized the Citation Overlay document as being from ChatGPT, and demanded all of the prompts Autenrieth used from 3M’s lawyers. The deposition was paused for three hours while they were gathered, and Moye was given 350 pages of ChatGPT conversations that Autenrieth had when creating the report. Those documents included ChatGPT’s public links to Autenrieth’s full conversations. Court transcripts suggest that 3M paid Knighthawk Engineering roughly $90,000 for its analysis, and a filing by 3M shows that Autenrieth’s rate was $475 per hour.
The conversations show much of Autenrieth’s process from start to finish, which included telling ChatGPT that he was “being retained as a professional expert witness by 3M in defense of them in their lawsuits and other legal proceedings behind the January 2020 explosion at Watson Grinding.” He told ChatGPT that he needed “to create an expert witness report to defend 3M’s standard of care for their work,” and that, specifically, it needed “to counter the defense witness [sic] outlandish and false claims particularly about working on equipment you are not trained to and without the right permitting.” He asked ChatGPT to help him find violations of various working standards, then attached hundreds of court records.
Autenrieth then asked ChatGPT to read all of the attached records and to defend 3M, “illustrating the lack of [Process and Safety Management] and safety by Watson Grinding, defeating [the defense witness’] comments […] and show how 3M is 0% at fault for the explosion at Watson Grinding and how my background and experience is well suited to render this professional opinion.”
ChatGPT created a roughly 30-page report that included the line “From a technical and standard-of-care standpoint, 3M is 0% responsible for the January 24, 2020 explosion.” This line did not make it into the final report filed with the court , because when Autenrieth later asked ChatGPT to “review this as the opposing council,” ChatGPT determined that writing “‘0% responsible’ is an easy target” for a lawyer to poke holes in, and is one of several "phrases [that] let opposing counsel paint you as an advocate rather than an expert."
In another chat , Autenrieth uploaded an image of a gas detector and asked ChatGPT “what am I looking at?,” and asked “what are model names of industrial gas detectors.” Moye said that the gas detector in question is “the subject of the whole case.”
Autenrieth used ChatGPT to help him make various edits to the report it had generated, and repeatedly uploaded different versions of the report, getting revisions from the tool, then uploading new versions of the report (in some of the chats he changed the subject from his expert witness testimony to having ChatGPT generate t-shirt images). He also asked ChatGPT to “grade” his report (it got a 97/100), and “what are the 5 main things in my report the prosecution could attack and how do I defend them?” He then asked ChatGPT if his resume was sufficient to be an expert witness; “will prosecution go after me for never having been [an expert witness] before based on wording and how do I defend that?”
The report that Autenrieth submitted to the court is structured the same as the initial output given by ChatGPT and large swaths of it are identical to what ChatGPT first outputted and the revisions that it recommended in Autenrieth’s subsequent chats.
“This expert relied on AI not as an assistive device, but exclusively relied on ChatGPT to form his opinions and write his report,” Moye told 404 Media in a phone interview. “He acknowledged [at trial] the prompts he put in were biased toward 3M to help 3M win the case […] it’s really egregious.” Moye added that many of the prompts took place the night before Autenrieth was deposed as an expert witness. “They hired him for the sole purpose of changing the outcome of the case. They hired him and he used ChatGPT to write these reports, so really, ChatGPT was the expert in the case. There’s just no question about that.”
Moye told 404 Media that 3M eventually tried to get Autenrieth disqualified from the trial, but that after he learned Autenrieth extensively used AI to generate his report, he took the somewhat unusual step of calling the other side’s expert witness as his own witness. “I said, I’m calling you to trial because I need a jury to hear from you because this is bad, bad stuff. And that’s exactly what I did,” Moye said.
In trial transcripts, Autenrieth admits to using ChatGPT, but said repeatedly that he is an expert in gas detection: “I've got a body of work and 20-plus years of experience in the industry.” He did not explain why he used ChatGPT, but said “my opinions were put in there, and AI helped me to draft a straw man to build off of,” and added “I put information and opinions in up front before it ever generated […] if the output wasn't of my opinion or what I agreed to, I did alter it.” In the examination at trial, Moye and Autenrieth agree that the submitted report is “90 to 85 percent ChatGPT.” Autenrieth also said that he prompted ChatGPT about the case several times while “sitting in traffic” to “jog my memory” about several different manufacturers of gas detectors.
Beyond this being a highly interesting case on its own merits, it shows that ChatGPT transcripts can be obtained by opposing lawyers in discovery or during depositions. Moye said “every lawyer needs to make sure their own experts aren’t generating work product in a way that’s insincere, and then knowing you can subpoena the prompts [...] I’ve got lawyers all over the place saying, 1) ‘Holy shit, man. How did you get the prompts?,’ and 2) ‘How many cases do I have where this is happening to us?’”
Autenrieth is not the first expert caught using AI during a trial, though most known cases have been international, according to a database of cases in which AI was used to create evidence. In a case in France, a lawyer used AI to make an argument about rainwater patterns in a flooding dispute; that evidence was thrown out. In Canada, AI was used to generate opinions about whether specific types of plants caused a bug infestation at an apartment complex. The judge in that case wrote “I place no weight on the [AI evidence]. Expert evidence must come from a known individual with known qualifications who has specific knowledge about the issue at hand.”
The jury in the 3M case awarded more than $61 million to the plaintiffs , apportioning 30 percent of the responsibility to 3M and 70 percent to Watson Grinding. Previous decisions awarded $118 million and 38 million to victims of the blast, and there are several more upcoming cases.
Knighthawk Engineering, Autenrieth, and 3M did not respond to a request for comment
About the author
Jason is a cofounder of 404 Media. He was previously the editor-in-chief of Motherboard. He loves the Freedom of Information Act and surfing.
GitHub is down for some users as a widespread outage is causing errors across the website, API, Actions, Pull Requests, and several other services.
GitHub confirmed the outage at 9:40 AM EDT on August 17, 2026, when it said it was investigating reports of performance problems affecting some of its services.
The problems quickly spread across several parts of GitHub that developers rely on, including API Requests, Actions, Webhooks, Issues, and Pull Requests.
According to GitHub's status page , the company is seeing error rates of around 20% across its web experience and API traffic.
GitHub says archive downloads and raw repository content downloads are experiencing error rates of approximately 50%.
Likewise, authentication-related services are also having problems, with SAML and OIDC authentication, SCIM, and Team Sync affected by the incident.
Some users are running into server errors when trying to access GitHub, while others are reporting problems loading commits, repositories, and Pull Request pages.
GitHub Actions is also experiencing degraded performance, which means the outage can affect automated builds, tests, deployments, and other workflows that depend on GitHub's CI/CD platform.
At 10:31 AM EDT, GitHub confirmed that Copilot was also experiencing degraded availability, expanding the outage to its AI coding services.
Git Operations, Packages, Pages, and Codespaces are currently listed as operational, but several important parts of GitHub remain degraded.
GitHub has not disclosed what caused the outage and says its investigation is ongoing.
This is a developing story...
Overall prevention scores can hide what happens after initial access. Once attackers are using valid credentials, prevention drops sharply.
The Blue Report 2026 measures defenses technique by technique across 338 million simulations run in customer production environments.
From Wednesday's decision in Phillips v. Parlade , by Judge Gloria Navarro (D. Nev.), where a litigant sued a state court judge in his case:
Plaintiff … argu[es] that judicial immunity does not apply in this matter because Defendant unlawfully delegated her official decision-making duties when she relied wholly on artificial intelligence to issue a judicial ruling, without any discretionary human thought, such that her actions cannot be considered a "judicial act." Plaintiff further argues that because Defendant delegated 100% of her decision-making duties, the rulings were in clear absence of all jurisdiction.
Judges enjoy absolute immunity from civil liability, even if their action was in error, done maliciously, or in excess of their authority. Judicial immunity applies unless the challenged conduct is accompanied by a clear absence of all jurisdiction or where the challenged conduct is not judicial in nature. Courts determine whether an act is judicial in nature by considering whether: (1) the act is a normal judicial function; (2) the events occurred in the judge's chambers; (3) the controversy centered around the case pending before the judge; and (4) the events at issue arose out of confrontation with the judge in his or her official capacity.
Here, Plaintiff alleges that Defendant issued a judicial decision in his state court case by relying wholly on artificial intelligence. Issuing a judicial ruling is clearly a normal judicial function and the controversy at issue centered around Plaintiff's state court case pending before Defendant. Moreover, there are no allegations that the events occurred outside Defendant's chambers. The challenged conduct is therefore judicial in nature. Furthermore, Plaintiff provides no case law or authority to support a finding that the challenged conduct was accompanied by a clear absence of all jurisdiction. Thus, Defendant is entitled to judicial immunity and this case must be dismissed.
Naturally, I can't speak to whether the allegations against the state judge are correct. But the federal decision in this case is that, as a matter of law, even if the allegations are correct and she had indeed relied entirely on AI in making her decision, she can't be sued for that in federal court.
Such objections to a state judge's actions can of course be raised on appeal to a state appellate court (or through various appeal-like remedies, such as petitions for a writ of mandamus or the like). And they can be raised in state court disciplinary proceedings. But, according to this case, they can't be raised in a federal district court lawsuit against the state court judge.
I gave Qwen3.8's MTP drafter another 69.2 MiB of precision. Throughput fell from 50.44 to 37.02 tokens per second. That result sums up the whole experiment: the best local inference setup is rarely made from the individually "best" parts.
I wanted a dense 27B model, its full 262,144-token context, multimodal input, maximum useful quality, and speculative decoding on an NVIDIA RTX PRO 4000 Blackwell SFF with 24 GB of VRAM. The server also had to survive real agent work after printing
model loaded
. The experiment followed a
hunch I had written about earlier
: careful operation may matter as much as moving to a larger model.
The finished system averages 50.44 tok/s in the current ten-run production series. On a strict runtime A/B, the custom llama.cpp build reaches 55.40 tok/s versus 45.42 for clean master, a 21.97% gain. Against target-only greedy decoding, embedded MTP moves 21.19 to 59.46 tok/s, or 2.81 times the throughput. At the far end of a genuinely occupied 256K cache, it still produces 12.61 tok/s without an out-of-memory failure.
Those numbers came from different gates and should stay separate. Combining them into one heroic speedup would make a better headline and a worse benchmark.
The winning setup came from the fit between the quant, drafter, CUDA kernels, memory layout, and workload. No component won on its own.
Qwen3.8 27B is a 64-layer dense model. Its repeating pattern contains three Gated DeltaNet layers followed by one full-attention layer, giving 48 recurrent layers and 16 conventional attention layers. It has a native 262,144-token context, a one-layer MTP head, and a separate 27-layer vision encoder.
The hardware is lopsided in a useful way:
Only the 16 full-attention layers grow a conventional KV cache with sequence length, which makes 256K less absurd than it first appears. With Q4 K and V, that cache costs roughly 4.25 GiB before allocator overhead. DeltaNet adds recurrent state and checkpoints instead. Four checkpoints were the useful minimum; the default 32 spent memory I needed elsewhere.
NVIDIA quotes 432 GB/s of peak bandwidth. That is a hardware ceiling rather than an application metric from llama.cpp, but it matters here. Autoregressive decode repeatedly streams quantized weights, and the 16 attention layers add increasingly expensive KV reads as context fills. This is why the same profile averages about 50 tok/s on the production task and 12.61 tok/s at the far end of a 261.5K-token cache.
The original plan was simple: estimate the capacity, select a quant, then benchmark it. The machine immediately taught me that capacity estimates are just admission tickets. The real test begins after loading.
I began with public GGUFs at 40K context. Q4_0 was surprisingly strong. Target-only decoding reached 22.40 tok/s, and MTP with
n_max=3
reached 44.95. It beat smaller Q3_K_M and nominally smarter Q4_K_M variants because file size and quant label do not describe the CUDA kernel that actually runs.
| Quant | Target only | MTP n=3 | Acceptance |
|---|---|---|---|
| Q3_K_M | 17.00 tok/s | 31.34 tok/s | 83.98% |
| IQ4_XS, iMatrix | 20.63 tok/s | 34.40 tok/s | 64.87% |
| Q4_0 | 22.40 tok/s | 44.95 tok/s | 80.40% |
| Q4_K_M | 17.57 tok/s | 26.15 tok/s | 66.86% |
Then quality testing spoiled the easy answer. On a short, identical WikiText-2 control, IQ4_XS scored 6.1175 perplexity while Q4_0 scored 6.3798. Q4_0 led the speed table. Hermes needed a main model, though, and that quality trade felt too expensive for a few hundred milliseconds. I would have been using a 27B model as oversized autocomplete.
The opposite extreme failed too. Q4_1 reached 6.1127 PPL, marginally ahead of IQ4_XS, but its memory footprint made 256K plus F16 vision uncomfortable. The useful point was somewhere between a fast blunt quant and a precise file that left no room for the rest of the system.
Early capacity tests looked excellent. Q4_0, MTP, Q4 KV, four recurrent checkpoints, and the F16 projector all allocated at 262,144 context. That still did not answer the question I cared about.
I filled the slot with 261,500 input tokens, generated another 256, and then reused the hot cache. No truncation. No OOM. The first Q4_0 profile decoded at 12.06 tok/s near the end of the cache, compared with 44.95 around 40K. GPU usage sat at 99 to 100%, while the server used roughly one CPU core. The bottleneck was the 16 full-attention layers reading a huge occupied KV cache, not a secret CPU fallback.
This changed the benchmark method for every run that followed. "262K loaded" was banned from the results table. A long-context claim had to include actual token fill, post-fill VRAM, hot decode, truncation state, and an output hash.
Blackwell has native FP4 hardware, so a ready-made NVFP4-MEDIUM GGUF looked like the obvious route. Its bulk target matrices used NVFP4, with a Q8 output head, Q6 embeddings, and an IQ4_XS MTP layer. It reached 40.46 tok/s and fitted the complete 256K plus vision profile with about 1,055 MiB free.
Its PPL was 6.4949. Worse than Q4_0.
The conversion recipe was the problem. Attention and DeltaNet weights from the source FP8 checkpoint had been expanded and requantized into NVFP4 along with the large, tolerant matrices. Native arithmetic made the file quick, while indiscriminate low precision damaged sensitive parts of the model. Hardware format support does not tell you where to spend the bits.
That failure gave us the design for a custom quant: use NVFP4 for the bulk, then protect only the tensors that our own workload says matter.
The calibration corpus started with 5,472 messages from 296 Hermes sessions. I placed that material before the generic corpus so the 153,600 processed tokens represented coding, Polish and English conversation, infrastructure work, tool calls, and the awkward mixtures my agents really see. A secret scan ran before calibration. No PEM keys, provider tokens, GitHub tokens, Slack tokens, or email addresses were present.
llama-imatrix
collected importance data for 497 target weights. NVFP4 does not consume an iMatrix directly during block quantization, so I used the matrix as a map: large tolerant tensors stayed native NVFP4; selected attention, DeltaNet, and FFN tensors moved to Q5_K or Q6_K; embeddings became Q6_K; the output head stayed Q8_0.
The first 5.14 BPW hybrid was the quality winner at 6.0967 PPL. It was also slow at 34.19 tok/s and too large to keep the desired projector on GPU alongside 256K. Good experiment. Bad production model.
The second build was tighter:
This became
Qwen3.8-27B-Hermes-iMatrix-NVFP4-Balanced.gguf
. It preserved the measured quality of the Q4_1 reference, ran faster, and left enough room for the actual serving stack.
The early sweep suggested
n_max=3
. Values 4 through 7 got slower as rejected draft work accumulated. Then n=8 jumped to 49.31 tok/s.
| MTP n_max | TPS | Acceptance | Combined process VRAM |
|---|---|---|---|
| 3 | 43.11 | 78.73% | 18,352 MiB |
| 4 | 37.79 | 62.22% | 18,502 MiB |
| 7 | 29.10 | 42.95% | 18,950 MiB |
| 8 | 49.31 | 48.33% | 19,100 MiB |
| 9 | 49.02 | 43.95% | 19,250 MiB |
| 12 | 43.31 | 34.34% | 19,700 MiB |
| 20 | 30.60 | 19.97% | 20,900 MiB |
The curve is jagged. Eight candidates hit a favorable batch and kernel shape. Nine was no faster, and each extra candidate cost about 150 MiB. With the full 256K allocation, n=9 at
ubatch=256
failed on one more 162 MiB CUDA graph buffer. Reducing ubatch to 128 made it load, but throughput fell to 52.70 tok/s and 32K prefill suffered. N=10 failed on another 81 MiB. Even an experimental scheduler pool lost to a 31 MiB allocation.
I kept n=8 because it was the last fast point before the allocator started biting.
I wanted a controlled rival for the NVFP4 MTP choice. A patched MTP-aware iMatrix run processed 300 Hermes-history chunks and added all eight MTP matrices. For the comparison, all 851 non-MTP tensors were verified byte-for-byte identical to production. Only the eight MTP weight tensors changed.
| MTP weights | Extra size | Mean TPS | Acceptance | Result |
|---|---|---|---|---|
| Production NVFP4 | baseline | 50.441 | 48.329% | keep |
| iMatrix Q5_K | 50.625 MiB | 48.733 | 46.751% | -3.39% |
| Q5_K with critical Q6_K | 69.219 MiB | 37.024 | 33.065% | -26.60% |
The higher-bit drafter may be closer to the BF16 source model. In production it had one job: predict this quantized target. The NVFP4 errors in the MTP head happened to align better with the NVFP4-heavy target, so its exact proposals survived more often. Standalone precision lost to quant-drafter alignment.
This changed how I treat drafter quality. The drafter and target form one quantized system, and their interaction decides acceptance and throughput. A standalone quality score for either half misses it.
I also tested Qwen3.8-27B-DSpark , a 1.36B diffusion drafter with a Markov head and confidence head. A llama.cpp patch allowed the sidecar to run on GPU1 while the target stayed on GPU0.
The best version was Q8_0 at 26.49 tok/s, 34.5% above its 19.70 tok/s target-only reference. Embedded MTP reached 49.61 tok/s on the same comparison. DSpark was 46.6% slower and used more aggregate VRAM. The sidecar had been trained against an FP8 target, while ours was a mixed NVFP4/Q5/Q6 target. PCIe traffic and the weaker Ada card did the rest.
DSpark's diffusion mechanism worked, but this sidecar was trained for a different target and lost on this box.
Once the model stabilized, I benchmarked runtime changes one branch at a time. N-gram speculation was disabled for every A/B because repeating a prompt taught the cache and pushed apparent throughput from 46.77 to roughly 180 tok/s. Useful in production, poison in a kernel comparison.
| Runtime | Measured result | Decision |
|---|---|---|
| Clean master b10454 | 45.422 tok/s | baseline |
| #26001 + #26048 + #26705 | 45.866 tok/s | keep, +0.98% |
| Add #27173 MTP chain | 55.402 tok/s | keep, +21.97% vs clean master |
| #27140 | 45.457 tok/s, prefill -1.59% | reject |
| #26079 | prefill -1.96%, hot decode -0.70% | reject |
The final build pins #26001 , #26048 , #26705 , #27173 , #24891 , and #25635 to audited commit hashes. The patches cover Gated DeltaNet, CUDA dispatch, faster Q4/Q5 speculative verification, chained MTP, recurrent-checkpoint correctness, and Flash Attention swizzling.
The Flash Attention patch deserves its own number. At 32K it moved prefill from 759.38 to 815.64 tok/s, up 7.41%, and hot decode from 37.26 to 38.23, up 2.61%. It helped where the attention workload was large enough to matter.
The builder refuses to continue if any PR head moves. It builds sm89 and sm120a, then runs sampling, quantization, Gated DeltaNet, Flash Attention, and NVFP4 matrix tests. Without those checks, the next upstream update could turn a fast private binary into an outage.
The active model is
Qwen3.8-27B-iMatrix-NVFP4-256K-MTP
. The important runtime settings are:
CUDA_VISIBLE_DEVICES=0,1
MTMD_BACKEND_DEVICE=CUDA1
LLAMA_SPEC_CHAIN=1
GGML_CUDA_GRAPH_OPT=1
llama-server \
--model Qwen3.8-27B-Hermes-iMatrix-NVFP4-Balanced.gguf \
--device CUDA0 --n-gpu-layers 999 --fit off \
--ctx-size 262144 --parallel 1 --ctx-checkpoints 4 \
--flash-attn on \
--cache-type-k q4_0 --cache-type-v q4_0 \
--batch-size 512 --ubatch-size 256 \
--temp 0.6 \
--spec-type draft-mtp --spec-default \
--spec-draft-n-max 8 --spec-draft-n-min 0 --spec-draft-p-min 0 \
--spec-draft-type-k f16 --spec-draft-type-v f16 \
--spec-draft-backend-sampling \
--mmproj mmproj-F16.gguf \
--reasoning-preserve --jinja --metrics
--spec-default
adds n-gram speculation in this build. It stays enabled because agent sessions repeat code, tool schemas, and prompt prefixes. It never appears in comparative benchmark numbers. Temperature is 0.6 in the stack, while reasoning effort remains request-controlled by Hermes. I do not force xhigh globally.
--agent
is deliberately absent. Hermes owns tools and MCP. Enabling llama-server's agent layer would duplicate the tool loop and widen the code-execution surface.
--fit off
is deliberate too: automatic fitting would silently change context or offload to reserve its default VRAM margin, invalidating the profile we measured.
The F16 projector uses 982 MiB on GPU1. That is better than squeezing it onto GPU0, and much better than running a separate 4B vision model. The active 17 GiB model and binaries stay on local NVMe for fast restarts; inactive experiments live on slower network storage.
| Gate | Result | What it means |
|---|---|---|
| Production, 10 runs | 50.441 tok/s mean, 49.420-51.397 | Current repeatable operating point |
| Strict runtime A/B | 45.422 to 55.402 tok/s | +9.980 tok/s, +21.97% |
| Greedy target vs MTP | 21.189 to 59.456 tok/s | +180.6%, 2.81× |
| Full 261.5K cache | 12.606 tok/s | Honest far-context decode |
| Full-context prefill | 226.750 tok/s | 261,500 input tokens |
| GPU0 during full fill | 23,952 / 24,467 MiB | About 515 MiB physical margin |
| GPU1 projector | 982 MiB | Vision remains resident |
The full-context run generated 256 tokens without truncation or OOM. Cold and both hot outputs had the same hash. The ten-run production series was also deterministic within its configuration.
One correctness caveat remains. Target-only greedy and MTP n=8 produce different continuations on a quantized target, matching the open llama.cpp batch-invariance issue #25618 . Both modes are internally deterministic and the outputs are coherent, but this max-TPS profile is not bitwise distribution-preserving relative to target-only decoding. N=1 is the safer lossless setting when that property matters.
I would start with the production task distribution and three hard gates: quality, real context fill, and deterministic A/B. Quant labels would come later.
I also learned to leave the last few hundred MiB alone. We intentionally ran near 76 MiB free during one tuning phase. It worked until a 31 MiB scheduler allocation and graph fragmentation showed why arithmetic free memory is not operational headroom. The final roughly 500 MiB margin keeps the server alive through graph creation, vision requests, and allocator variation.
Next time I will test the drafter against the exact quantized target from day one. The 69 MiB "upgrade" settled the question: the higher-precision MTP weights reduced agreement, acceptance, and throughput.
Synthetic benchmarks only cover part of the system, so I gave the final Qwen setup ten one-shot browser tasks through Cursor Agent Local. "One shot" means the agent received the prompt once in an empty directory. There was no human feedback, second attempt, or repair pass after the answer. It could plan, write files, inspect them, and test its own work during that single run. The result therefore measures the model and the harness together.
Below are five untouched artifacts, ordered as I would show them to another engineer.
A procedural Japanese garden built from voxel geometry, with a multi-level pagoda, torii, cherry trees, lanterns, particles, shadows, and orbit controls.
31m 01s 686 lines 26,359 bytes
A 100,000-particle GPU scene with six morph targets and webcam hand controls for scale, rotation, collapse, explosion, and trails.
60m 00s 966 lines 36,504 bytes
A colorful procedural island with a controllable character, NPCs, collectibles, houses, moving water, clouds, vegetation, and a following camera.
56m 19s 1,049 lines 42,992 bytes
A single-file Counter-Strike-inspired FPS with weapons, recoil, bots, bomb logic, a buy menu, minimap, particles, lighting, and a full HUD.
60m 00s 1,732 lines 73,128 bytes
A playable volumetric Tetris board with three-axis rotation, disappearing planes, orbit controls, shadows, particles, score, preview, and game-over state.
48m 51s 724 lines 27,685 bytes
Adapted and expanded from a field note originally shared on LinkedIn · August 17, 2026. Read more analysis on the blog or connect on Substack .
Meet Foreman , an eve software factory that puts AI agents on every stage of the development loop and keeps people on the judgment calls.
Foreman takes tasks from GitHub and Linear, moves each one through four stations, and delivers a reviewed draft pull request on your repository. You review, mark ready, and merge.
Each station is its own agent with its own instructions, sandbox, and tools. The Reviewer sees only the pushed branch, never the Implementer's reasoning. Between runs, Foreman keeps a factory brain : notes about your repository that every run starts from. See the pipeline and factory memory for the full picture.
factory
.
The pipeline runs on its own, posts progress as stations complete, and ends with a draft PR linked to the issue.
The Vercel deploy flow sets up everything: the
GitHub
connector,
Linear
connector,
Vercel Blob
store, and a prompt for the
FACTORY_REPO
and
FACTORY_LABEL
environment variables.
Configuration (see
.env.example
):
| Variable | Required | Default | What it does |
|---|---|---|---|
FACTORY_REPO
|
Yes | — |
The
owner/repo
the factory works on (the build fails without it)
|
FACTORY_SETUP_COMMAND
|
No | — |
Runs once inside the sandbox checkout at build time (e.g.
pnpm install
), so every run starts with dependencies already installed
|
FACTORY_LABEL
|
No |
factory
|
The issue label that hands an issue to the factory |
FACTORY_BRANCH_PREFIX
|
No |
factory/
|
Branch prefix marking the factory's own PRs, which are the only branches automated CI fixes touch |
FACTORY_BOT_NAME
|
No | the GitHub App's slug |
The
@mention
name, resolved from the connector automatically when unset
|
GITHUB_CONNECTOR
/
LINEAR_CONNECTOR
|
Yes | — | Set automatically from Vercel Connect connector UIDs |
Link the project you deployed (or a fresh one), pull its environment, and start the TUI:
vercel link vercel env pull pnpm dev
Hand the agent a task ("users report the password reset email arrives twice, fix it") and watch the four stations fire in order, ending in a draft PR on
FACTORY_REPO
. Local runs are treated as untrusted, so changes to GitHub wait for your approval in the TUI.
a large boulder the size of a small boulder
“It highlights the brutality of medieval warfare,” said paleopathologist Jo Buckberry.
This still from The Wolf at the Door, by Bob Marshall of Historic Environment Scotland, shows what the siege of Stirling Castle might have looked like, complete with trebuchets. Credit: Bob Marshall and HES
This still from The Wolf at the Door, by Bob Marshall of Historic Environment Scotland, shows what the siege of Stirling Castle might have looked like, complete with trebuchets. Credit: Bob Marshall and HES
During a medieval siege, a trebuchet scored a chance hit one unlucky defender. A small boulder slammed into his upper back at close to 300 kilometers an hour, smashing bones and crushing him beneath its weight.
The man’s skeleton lay beneath the chapel of Scotland’s Stirling Castle, along with the remains of several other people who had clearly died violent deaths. But this one, known to us only as Skeleton 150 (or as “trebuchet guy” in your faithful correspondent’s notes), stood out even among that battle-damaged crowd because most of his upper body had been shattered. According to University of Bradford paleopathologist Jo Buckberry, who presented her research at the 25 th European Meeting of the Paleopathology Association last week, skeleton 150 is the only known trebuchet casualty in history.
Skeleton 150 was in terrible shape, even for a dead guy. The man’s skull had broken in 61 separate places, and another 60 jagged bits of bone were distributed across his ribs. His right shoulder was broken, and his right leg above the knee was basically shattered.
Buckberry and her colleagues threw everything they could, metaphorically speaking, at the jigsaw puzzle of broken bones: forensics, microscopes, X-rays, and micro-CT scans. All of those results pointed to a single moment of crushing impact by something heavy and fast-moving. The pattern of the injuries suggests that the man’s right shoulder and the back of his head caught the brunt of the impact, while his ribs probably cracked under the weight of something heavy slamming him into the ground and pinning him there.
The bones had broken when they were fresh, so it was clearly not a case of “oops, we dropped a big rock on the corpse.” And the amount of force required to do so much damage in a single, shattering blow ruled out several messy possible ways to die during a medieval siege: Skeleton 150 hadn’t been trampled by a horse or run over by a cart, for example. Even if he had fallen from the castle walls, he probably wouldn’t have landed with enough force to do all of that, and the angle would have been all wrong.
“The most similar cases I was finding were car crashes and people hit by trains,” Buckberry told Science.org’s Andrew Curry . Assuming Stirling Castle hadn’t been attacked by time-travelers, though, that raised new questions. “What is large and moving very quickly in 1304?”
Big rocks flung by trebuchets, that’s what.
A trebuchet works a bit like the world’s deadliest see-saw; at the long end of the see-saw is the sling that holds the projectile, and at the shorter end is a counterweight (a box of rocks or lead, 10 to 100 times heavier than the projectile). When the counterweight drops, its momentum raises the other end of the see-saw, tossing the sling up in a big arc. And when the sling reaches the end of its length, it stops—but the projectile keeps flying until it hits something.
A typical trebuchet of the time could launch a 90-kilogram projectile (typically a big rock) a few hundred yards. According to this handy medieval trebuchet calculator , that means the projectile would have smashed into Skeleton 150 at somewhere between 200 and 300 kilometers per hour, depending on the exact weight of the projectile and the trebuchet’s counterweight, the length of the sling, and the airspeed velocity of an unladen swallow. That’s an impact with a little over 200 kilojoules of energy.
Trebuchets and other siege engines (we will, under no circumstances, be delving into the distinctions between the different types of catapult ) were meant to take out walls and other fortifications. They weren’t antipersonnel weapons; people undoubtedly got injured by flying debris or collapsing structures, but medieval engineers weren’t aiming their catapults at individuals. Skeleton 150 just got terribly unlucky.
“While the use of siege engines is well-documented, we believe this is the first evidence of trebuchet trauma in the archaeological record,” wrote Buckberry, adding, “The lack of directly comparable forensic cases remains a limitation.”
Please do not, though.
Credit: Kiona Smith
Please do not, though. Credit: Kiona Smith
Skeleton 150 was one of nine people found in graves beneath the castle’s medieval chapel, to the surprise of renovation workers, in 1997. Five of the dead had wounds that suggested they had died violent deaths: a middle-aged woman had suffered two heavy blows to the side of her head before falling and being struck with a war-hammer, which left a pair of eerily-neat square holes in the top of her skull. A teenage boy had been stabbed in the chest with a sharp weapon that left its mark on his ribs, then struck hard in the jaw, collarbone, and torso.
It’s hard to say exactly who any of these people were, besides obvious medieval wartime casualties. All five of their remains radiocarbon dated to around the Scottish Wars of Independence in the early 1300s, during which Stirling Castle was hotly contested real estate; it changed hands five times just in the eight years between 1296 and 1304, and the wars carried on until 1357. So some of the people buried in the chapel may have been Scottish, and others may have been English.
At least one of the dead, whose skeleton bore the marks of a lifetime of healed wounds, turned out to be an English knight, Sir John de Stricheley , who died in 1341. The ratio of chemical isotopes in his bones suggested that he had grown up eating food grown on the bedrock of southern England, and that, combined with his age and apparent status, pointed to an English knight. Sir John de Stricheley died at the castle in 1341 (and was important enough to have his death written down properly).
Buried not far from Sir John and the others lay skeleton 150, who Buckberry and her colleagues say was most likely a Scottish defender of the castle killed during the siege of 1304, when King Edward I of England laid siege to Stirling Castle with what Historic Environment Scotland describes as “possibly the largest array of siege engines ever assembled by the kingdom of England.”
In 1304, Skeleton 150 was probably one of the 25 men, led by Sir William Oliphant, who made up the last gasp of Scottish resistance against England and Edward I. After defeating William “They Will Never Take Our Freedom” Wallace at the Battle of Falkirk in 1298, Edward spent the next six years conquering the rest of Scotland. The country’s nobles eventually surrendered in exchange for being allowed to keep their lands (some of them may have done so with their fingers crossed ). Edward’s long and bloody campaign was nearly over, except for this one castle, which inconveniently happened to perch along a strategically important road to Edinburgh. And the king was not amused.
In preparation for the siege, Edward decided to flex his newly won power over the Scots, which was undoubtedly as much a political gesture as a strategic choice. He ordered the Scottish nobles who had just surrendered to send him troops and horses for the siege, and he ordered that every church in Scotland strip the lead from their roofs and send it to the siege as well. Thirty-nine carts of lead arrived from around Scotland, mostly from St. Andrew’s, along with “all the iron and great stones of Glasgow.”
Thirteen siege engines were in place when Edward I arrived on the scene in April of 1304, and they bombarded the castle around the clock for three solid months. Somewhere during those months, Skeleton 150 met a bad end.
The War Wolf, probably not the trebuchet that killed Skeleton 150.
Credit: Bob Marshall, HES
The War Wolf, probably not the trebuchet that killed Skeleton 150. Credit: Bob Marshall, HES
As the siege wore on, Edward I set his engineers and carpenters to work on the largest trebuchet anyone in the world had ever seen. He dubbed it the War Wolf, or Loup de Guerre, and it was an absolute monster—so much so that Oliphant, who had held the castle for three months and still had enough men and provisions to keep antagonizing the king for a while yet, decided that actually, he would very much like to surrender after all, before the giant trebuchet took the field, thank you very much.
And Edward I told him no.
The king was not going to miss the chance to use his tremendous new trebuchet, nor the chance to very publicly and definitively (and also literally) crush the last of the Scottish opposition to his reign. So he ordered his engineers to load the War Wolf and fling a 140-kilogram projectile straight through the castle’s curtain wall. Witnesses compared it to watching an arrow pierce a paper target.
It’s tempting to speculate that Skeleton 150 might have been a casualty of the infamous War Wolf itself, but that seems unlikely. For one thing, the odds just aren’t in favor of the theory; the War Wolf flung a single, albeit enormous, shot on the very last day of the siege. Thirteen other siege engines had been firing at all hours of the day and night for three months before that.
He was probably dead and hastily buried in the chapel by the time the castle walls crumbled and Oliphant and the other survivors were rounded up and taken south to prison in England.
Kiona is a freelance science journalist and resident archaeology nerd at Ars Technica.
A native Mac terminal that keeps projects, sessions, and attention organized.
Run shells, dev servers, tests, and coding agents across several projects while Saggar tracks what is working, waiting, finished, and failed. Sessions that need a decision stay visible without pulling you away from the terminal you are using. Pair the Companion to check in and respond from your phone while you are away.
Mac only. Requires macOS 26 Tahoe or later on Apple silicon.
One agent is editing while another runs tests. A third has quietly stopped at a permission prompt, and a fourth finished ten minutes ago with changes to review. In a normal terminal, they all look like tabs. You find the blocked one when you happen to look.
Saggar keeps every state visible, folds away the quiet work, and builds one queue from everything that needs a decision.
Run agents, shells, dev servers, and tests across every codebase. Each session stays attached to its project, branch, and worktree.
Sessions classify themselves as needs you, working, idle, finished, or failed. Quiet work folds away while the claims on you stay visible.
Waiting prompts and finished work form one ordered queue. Answer from the attention card, or press ⌘J to walk it from loudest to quietest.
Pair the Companion to see what your sessions are doing while you are out. Inspect a terminal, answer its prompt, interrupt a runaway, or type into it.
Remote control ships on, because a feature you have to find and arm first is one nobody has armed by the time they need it. What says no is the account: nothing pairs that is not signed in as you.
your account, both ends
A device pairs only while it is signed in as you — checked on every request.
nothing listens
The Mac dials out to the relay. It opens no port on your network.
one-time pairing
One scan, and it holds. Signing the Mac out revokes every device at once.
sealed traffic
Terminal requests and responses are encrypted end to end through the relay.
Start with the setup guide, learn the common workflows, or look up a command when you need it.
Follow the setup guide, or open the remote client for a paired Mac.
You might already know that atoms are mostly empty space. Physical size behaves a bit weirdly at the atomic scale, but essentially the radius of an atom’s nucleus is tens of thousands of times smaller than the radius of an electron’s orbital. That’s the same ratio as a single grain of sand 1 to a football field. 2
So what can we fit in all that space? Well, if you build it just right, you can fit a bunch more atoms of the same element – theoretically over a hundred. Yes, scientists actually did this, and no, it doesn’t violate the laws of physics. You just need one really big atom, and a bunch of really cold normal ones. 3
Let’s start with the big atom. An atom consists of a nucleus and the electrons that surround it. So if you want to make an atom bigger, you can move the outermost electron farther out from the nucleus. This is called “electron excitation”, and it happens when an electron absorbs extra energy and temporarily jumps to a farther out, more loosely bound orbital. This happens all the time: atoms in nature are constantly absorbing and releasing the energy around them.
As an electron gets bumped to higher and higher energy levels, the radius of its orbital gets bigger very quickly. But if we add too much energy, the electron flies away from the nucleus completely, and we’re left with an ion – not a complete atom. So we need to tune the energy we add very carefully. The biggest atoms we can create like this are called “Rydberg atoms” after Swedish physicist Johannes Rydberg. In 1888, Rydberg discovered a pattern in atomic spectra which would later lead to the theory of electron energy levels. In laboratories, Rydberg atoms can be over 1000 times wider than a normal atom – plenty of space to stuff some more atoms inside.
Johannes Rydberg: Not an atom, but made of atoms
But if the stuffed-in atoms have too much energy of their own, they’ll knock that carefully placed electron away. So we need to get rid of nearly all of their energy. For that, we can turn to an exotic state of matter called a “Bose-Einstein Condensate”, or BEC for short.
Remember that temperature is a measure of particles are bouncing around, so if you cool particles down you can get them to move slower and pack closer together. For certain atoms, at well under a millionth of a degree above absolute zero, a cluster of atoms will start to all share the same quantum state. This has all sorts of cool scientific implications, since it lets quantum physics researchers see quantum effects on a much larger scale, but for our purposes all we need to know is that the atoms in a BEC get really, really cold.
Density map of a BEC forming over time (left to right)
So in 2018, an international team of scientists created a BEC out of strontium atoms, and hit one of those atoms with a carefully tuned laser, exciting its outermost electron and turning it into a Rydberg atom. Several other atoms from the BEC were caught within between that outer electron’s inflated orbital. Those interior atoms interacted slightly with the electron, and together formed something like a very bizarre molecule, called a “Rydberg polaron”. 4 And according to the scientists’ computer simulations, up to 170 more strontium atoms could pack themselves into the Rydberg polaron.
So yes, you can fit a bunch of atoms in an atom. It’s unclear right now if Rydberg polarons will ever be directly useful to ordinary people, but they certainly have a lot to teach us about quantum mechanics and the way atoms work. And sometimes, that’s all the justification you need.
—
I think we’re pretty good at telling the difference between an AI generated image and a real photograph, but when I need help I call Hany Farid.
Farid is the cofounder of GetReal, a company that specializes in detecting deepfakes and other AI-generated images, and is the developer of PhotoDNA, a perceptual hashing algorithm that helps companies automatically detect and remove some of the worst images that exist online, and that is now being used by every serious internet platform in the world.
We started talking to Hany regularly in 2017, when Sam first reported on deepfakes. As that technology evolved and changed how we perceive reality, so have our conversations. I wanted to talk to him today on the podcast so you could hear of those conversations, and so that Hany and I could zoom out, reflect on the past few years, and speculate about where we might be headed.
404 Media is a journalist-founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers get access to additional episodes where we respond to their best comments. Subscribers also get early access to our interview series. Gain access to that content at 404media.co .
Listen to the weekly podcast on Apple Podcasts , Spotify , or YouTube .
Become a paid subscriber for early access to these interview episodes and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.
About the author
Emanuel Maiberg is interested in little known communities and processes that shape technology, troublemakers, and petty beefs. Email him at emanuel@404media.co
As part of ongoing security research conducted through Snowflake’s HackerOne vulnerability disclosure program, Wiz Research’s "Red Agent"—an autonomous, AI-powered security research tool—identified a critical GitHub Actions workflow vulnerability in one of Snowflake’s public repositories.
This incident highlights a rapidly emerging reality in software development: how AI coding assistants can inadvertently introduce workflow injection vulnerabilities, and how automated AI agents can rapidly surface them in the wild.
Upon responsible disclosure on June 23, 2026 by Wiz, Snowflake remediated the vulnerability on the same day, rotated the affected credential, and verified via detailed audit logs that Wiz was the sole actor during the exposure window. Wiz confirmed that all data accessed during proof-of-concept testing was securely deleted.
Wiz Red Agent identified a script injection vulnerability in snowflakedb/snowflake-connector-net . The issue allowed an unauthenticated user to execute arbitrary commands within a GitHub Actions runner by opening a GitHub issue with a specially crafted title.
Crucially, the vulnerability was introduced on June 18, 2026—just five days prior to discovery—via a commit co-authored by
Copilot Autofix powered by AI
(
PR #1218
). The AI assistant removed the repository's existing sanitized input pattern and replaced it with direct string expansion in a shell script.
Wiz Red Agent's CI/CD capability scanned Snowflake's GitHub organization and flagged the
jira_issue.yml
Workflow in
snowflakedb/snowflake-connector-net
as vulnerable to script injection via untrusted input in
run:
blocks.
AI Assistant (Github Copilot) Change
The workflow triggered on
issues: opened
- meaning any GitHub user could fire it by opening an issue - and interpolated the attacker-controlled issue title directly into a shell script:
The
sed
escaping runs after GitHub's template expansion, a single quote in the title breaks out of
echo '...'
and allows arbitrary command execution.
The injectable pattern was introduced just days earlier, on June 18, 2026,
commit 4a1b8ce
(
PR #1218: “SNOW-2069227: Update jira workflows”
) - co-authored by
Copilot Autofix powered by AI
.
It removed the repository’s existing safe pattern, which passed the issue title through an
env:
variable and built the JSON payload with
jq
. Instead it used the direct
${{ github.event.issue.title }}
interpolation shown above. In other words, an AI “autofix” commit created the very injection vector.
The workflow had an
if:
condition that appeared protective:
However, on issues events,
github.event.pull_request
is always
null
.
So the condition reduces to (
null != 'whitesource-for-github-com[bot]'
). This is always true, and every GitHub user passes the gate.
We crafted an issue title that, after template expansion, breaks out of the echo string and exfiltrates the Jira credentials via an out-of-band callback:
Crucially, when Red Agent’s cicd capability initially attempted exfiltration using a standard comment character (
#
), the runner returned a bash syntax error because the comment consumed the closing parenthetical of
TITLE=$(...)
. Rather than stopping or failing, Red Agent:
autonomously analyzed the syntax execution error
adjusted its payload to use
; echo '
to properly close the shell block, and
successfully received the out-of-band callback
Within seconds, our listener received the callback from a GitHub Actions runner (Azure IP
20.106.182.197
) containing base64-encoded credentials.
Note:
Our first attempt used
#
to comment out the rest of the line, which caused an unexpected EOF bash error because it also ate the closing
)
of
TITLE=$(...)
. The fix was using
; echo '
to properly close the shell syntax.
The exfiltrated token authenticated as
qa@snowflake.net
to
snowflakecomputing.atlassian.net
, granting read access across Snowflake's engineering, security compliance, and bug bounty tracking projects.
Same-Day Patching:
Snowflake patched the workflow on June 23, 2026 (
1dc7766
, PR #1402), fully restoring the safe
env:
variable and
jq --arg
parsing pattern.
Credential Revocation: The JIRA token in question was revoked and rotated.
Forensic Verification: Comprehensive audit log analysis confirmed that no external third parties accessed the endpoint during the 5-day exposure window. All anomalous queries were strictly matched to Wiz's testing IPs.
AI Code Generation Demands Rigorous Oversight: AI coding tools predict code based on probabilistic patterns, which can inadvertently reintroduce deprecated or insecure shell patterns. AI-generated PRs must undergo the same static analysis and security scrutiny as human code.
Collapsing Discovery Windows: The vulnerability was live for only five days before an automated agent discovered and validated it. Security operations must adapt to a landscape where automated discovery occurs in hours, requiring rapid patch cycles and short-lived credentials.
Preventing AI Security Regressions: Automated AI assistants often lack historical context regarding why specific code patterns were chosen. In this incident, an automated PR removed a safe env: + jq parsing pattern that had been explicitly implemented to prevent shell injection. Security teams must implement Guardrails that block AI agents from replacing structured data parsers with direct string interpolation.
June 18, 2026 -
Script-injection pattern introduced in
jira_issue.yml
by commit
4a1b8ce
(PR #1218), co-authored by
Copilot Autofix powered by AI
June 23, 2026 - Wiz identified, exploited, and reported vulnerability to Snowflake via HackerOne (report #3819931)
June 23, 2026 - Slack notification sent to Snowflake security team
June 23, 2026 (same day)
- Snowflake patches the vulnerable script-injection workflow (
commit 1dc7766
,
PR #1402
), restoring the safe
env:
+
jq --arg
pattern.
June 24, 2026 - Jira token rotated
July 25, 2026 - Public disclosure deadline (30 days after the June 25 resolution, per Snowflake’s disclosure policy)
Snowflake appreciates Wiz's responsible reporting of and collaboration around these findings through our vulnerability disclosure and bug bounty program, HackerOne. Wiz Research reported a security vulnerability in one of Snowflake's public GitHub repositories. The disclosure was received on June 23, 2026, and it was immediately investigated and remediated, and our investigation found no evidence of unauthorized access. Protecting our systems remains a top priority, and we remain committed to continually strengthening our software development and security practices. We are working together with Wiz to share these learnings with the broader industry to encourage widespread adoption of these security best practices.
The same countries split three ways, by how often people actually get them wrong. Start easy and work up — each level keeps its own progress.
The same games applied to the geography that is not a country — states, rivers and oceans.
Ten ways to play the same countries. Switch any time — your progress follows you.
Hardest flags to guess
The flags people get wrong most often — obscure designs with close look-alikes. Ranked, with what makes each one difficult.
Flags that look the same
Chad and Romania, Monaco and Indonesia, Ireland and Côte d'Ivoire. Every confusable flag pair, with the detail that separates them.
Red, white and blue flags
All the national flags using red, white and blue — far more than most people expect, and a common source of quiz mistakes.
Newest flags
The most recently adopted national flags, from the newest design backwards, with the year each one came into use.
Oldest flags
The oldest national flag designs still flying today, ordered by the year each was adopted.
Flags with animals
Every national flag featuring an animal — eagles, lions, birds of paradise and one dragon. With what each animal represents.
Every quiz here runs on the same set of 197 countries and territories, with flags, capitals, land area, population and borders drawn from one dataset. Pick a continent or a smaller region and the questions narrow to it.
Wrong answers are chosen deliberately rather than at random. A question about Chad will offer Romania; a question about Monaco will offer Indonesia. Guessing by elimination does not work, which is the point — those are exactly the pairs people get wrong in real life.
When a round ends you get a list of what you missed, and each entry links to a page about that flag: which colours it uses, when it was adopted, its proportions, and the flags it is most often confused with.
There is no account and no score history on a server. Your per-item accuracy is kept in your own browser so flashcard mode can lead with the items you are weakest on, and nothing is sent anywhere.
Start with the world flags quiz above. It draws from all 197 countries and shows you immediately which regions you are weakest in.
Yes. The map quizzes support pinch-to-zoom and drag, and every other mode is designed for one-handed use.
Every result screen lists what you missed, and each entry links to a page explaining that flag: its colours, its history, and the flags it is most often confused with.
Apple will change its rules on how app providers can use user data on iPhones and iPads for personalised advertising. The Bundeskartellamt objected to the way in which Apple had designed different consent requests for Apple’s own offerings and third-party apps. Apple considers its rules (set out in its so-called “Apple Tracking Transparency Framework”, ATTF) to be compliant with competition law; nevertheless, the company offered commitments which the Bundeskartellamt has now declared binding. The proceeding has thus been concluded.
Apple’s ATTF introduced rules for third-party app providers on the use of data on iPhones and iPads. For specific forms of cross-company data use, third-party app providers must obtain not only user consent under data protection law, but also additional consent through a prompt that is predefined by Apple. However, these ATTF rules do not apply to Apple’s own offerings; Apple uses user data from its own ecosystem and therefore its own prompt to request user consent to personalised advertising.
Andreas Mundt, President of the Bundeskartellamt:
“
It is key that personal data and privacy are protected effectively when using apps. Apple is allowed to provide for a level of protection for its users that exceeds the minimum legal requirements. However, if Apple sets up additional rules within its ecosystem for the use of data, these rules must, under Germany’s special abuse provision for large digital companies, not treat its own offerings better than those of its competitors. This is precisely where our competition concerns arose. Apple will now align the consent requests much more closely and give third-party app providers more freedom to combine the necessary requests in a sensible way
.”
Many third-party apps are, at least partly, funded through advertising. Personalised advertising can generate higher revenues for app publishers. Other apps are funded through user payments, for example for the purchase of the app or subscriptions. In these cases Apple often receives a commission, whereas Apple generally does not receive a share of the app publishers’ advertising revenue. As a general rule, personal data may in any event only be used for advertising purposes if users give their consent in accordance with the requirements of German and European data protection law.
Apple argued in the proceeding that the ATTF is meant to protect user privacy and that it is a competition law-compliant measure that also helps Apple position itself as providing a particularly high level of data protection. By contrast, the associations admitted to the proceeding, representing the branded-goods, media and advertising industries, took the view that, being a powerful gatekeeper, Apple was not allowed to set up additional, “extra-statutory” rules in the first place if these rules restrict other companies in their business activities.
In the Bundeskartellamt’s preliminary assessment, competition law generally also allows powerful companies such as Apple to take measures to protect their users’ privacy. However, the differences between the consent request used for Apple’s own offerings and the consent request predefined by Apple for third-party apps exceeded what could be justified based on differences in types of data processing. The wording, design and selection options of the request used for Apple’s own offerings had the potential to encourage users to give their consent, whereas they had the potential to discourage consent for third-party apps. In addition, third-party apps in some cases had to request consent several times even when users had already given data protection law-compliant consent.
With its operating systems and its App Store, Apple controls a key infrastructure for the distribution of apps on its devices. In addition, Apple offers its own apps and advertising space. This dual role makes Apple subject to specific competition law requirements. In the Bundeskartellamt’s preliminary assessment, there was a risk that, by setting out different rules, Apple was favouring its own offerings and impeding third-party app publishers.
Apple will modify consent prompt and simplify consent requests
Under the commitments that have now been declared binding, Apple will align the consent prompts for its own offerings and for third-party apps much more closely. This involves removing possibly discouraging symbols and wording in Apple’s predefined requests for third-party providers. The design of the consent prompts will be neutral in terms of content, wording and layout. In addition, app publishers and content providers, such as media publishers, will be given more scope to explain to users what significance personalised advertising has for their offering and their business model.
Under the commitments, Apple will also reduce the complexity of the current consent request architecture for third-party providers. In particular, app publishers will be given more freedom to combine the consent request required by Apple with the consent requests required under data protection law or connect them in a way that is clear to users. The improved conditions may also benefit advertisers and technical service providers to the advertising industry.
Andreas Mundt:
“
It is expressly not our aim to help achieve the highest possible levels of consent to personalised advertising. We want to ensure that users can make a free and informed decision. Users who do not wish to allow their data to be used for personalised advertising must be able to make an equally free and informed decision as users who intend to consent to such data use. The new consent requests are aimed at better enabling users to make this decision.
”
The Bundeskartellamt’s proceeding only examined whether Apple was in violation of German or European competition law. It did not aim at enforcing data protection law. To avoid possible delineation issues with data protection law, the Bundeskartellamt exchanged views with the Federal Commissioner for Data Protection and Freedom of Information (BfDI) and the Bavarian State Office for Data Protection Supervision (BayLDA).
Cooperation with other European competition authorities
Competition authorities of other EU Member States have also conducted proceedings concerning ATTF, some of which have already been concluded. To ensure that European competition law is applied consistently, the Bundeskartellamt maintained close and constructive dialogue with the relevant European authorities and the European Commission within the European Competition Network (ECN) throughout the proceeding.
In two proceedings by other European competition authorities concerning ATTF, the authorities have already imposed substantial fines on Apple. Last year, the French and the Italian competition authorities imposed fines on Apple totalling 150 million and 98.6 million euros, respectively. The Bundeskartellamt’s proceeding aims at achieving that the future design of the ATTF complies with competition law. The solution now achieved in Germany forms part of this European dialogue and may also affect the future design of the ATTF in other EU Member States.
Special abuse control of large digital companies
The Bundeskartellamt’s proceeding was based on, in particular, Section 19a of the German Competition Act (GWB) and the prohibition of abuse of a dominant position under Article 102 TFEU. Section 19a GWB gives the Bundeskartellamt special powers of abuse control of large digital companies that are found to be of paramount significance for competition across markets. In a first step, the Bundeskartellamt issues a decision declaring that a company has this special competitive position. In a second step, the authority may prohibit the company from engaging in certain anti-competitive conduct.
The Bundeskartellamt issued a decision finding that Apple is of paramount significance for competition across markets in April 2023. The Federal Court of Justice confirmed this decision in March 2025 .
Course of the proceeding
The Bundeskartellamt initiated the proceeding against Apple in June 2022. In February 2025 the authority informed Apple and the associations admitted to the proceeding of its preliminary legal assessment (see press release of 13 February 2025 ). Later in the proceeding, Apple offered commitments, which the Bundeskartellamt assessed in a market test in December 2025 (see press release of 2 December 2025 ). After further amendments to the commitments, the Bundeskartellamt has now declared them binding and concluded the proceeding.
Apple has four months from service of the decision to implement the changes proposed in the commitments and, before implementation, will test them together with app publishers. The commitments apply for seven years and will be monitored by an independent monitoring trustee.
Further details on the proceeding, the changes to the ATTF and their effects can be found in the accompanying FAQ document .
One of the biggest questions I get at Drop-In Time at the library (besides “what is taking up all my cloud storage?”) is how to disable or avoid intrusive AI that shows up where people don’t want it. This is a guide for people who would like less intrusive AI in their tech environment. Maybe you like AI and find it useful? That’s fine, this document is probably not for you. Additions/edits welcome. Leave a comment or drop me an email. This page is available at the short URL https://NoToAI.org
Uncheck the checkbox on the page, click Save .
Choose Disable new Acrobat Reader from the top menu (upper left), approve the dialog box and restart.
Depending on your phone’s manufacturer, you may be able to uninstall the Gemini app entirely. If not, there are some application-specific ways you can turn off some of its features
Tap your account picture, select Messages settings , then Gemini in Messages , and toggle the assistant off.
Tap your profile icon, select Gemini Apps activity , and then choose Turn off or Turn off and delete activity . Next, tap the profile icon again and go to the Connected Apps setting (check the Personal Intelligence setting). Disable all the apps where you don’t want Gemini.
If Gemini has “taken over” your power button, you can turn this off. Go to Settings, then System , then Gestures and change the settings under Press & Hold Power Button
If that doesn’t work you can try opening Settings , then searching for Power key
Only exists on iPhone 16 and newer Macs and iPads Open Settings (iPhone or iPad) or System Settings (Mac) and choose Apple Intelligence & Siri . Then turn off the Apple Intelligence option. Confirm your choice in the dialog that appears by tapping Turn Off Apple Intelligence . More details on how to turn off specific parts of Apple Intelligence and leave others alone .
Apple has a “learn from this application” feature which can be on even when Apple Intelligence is not on. To turn it off
For Mac: go to
System Settings
–
Apple Intelligence & Siri
. Select
About Siri, Dictation & Privacy…
near the end of the page, turn off apps you don’t want Siri to learn from
For iOS:
Settings
–
Apple Intelligence & Siri
, scroll down to
Apps
, disable “Learn from this App”
Type chrome://flags into the address bar and hit Enter . You’ll see a list of system flags and a search bar; look for GLIC [Google live in Chrome]. This will filter the massive list down to about a dozen AI features. The second search term you’ll need in this window is “ Gemini ” set them all to “disabled.” Don’t accidentally enable the couple of settings which default to disabled.
Similar to Chrome: type edge://flags into the Edge address bar, hit Enter , then type “ AI ” or “ Copilot ” into the search box.
Click Appearance in the left-hand Settings sidebar, and scroll down to Copilot and sidebar
Turn the sidebar off , and turn off the “ Personalize my top sites in customize sidebar ” and “ Allow sidebar apps to show notifications ” toggles.
Click Copilot under App specific settings . Turn off “ Show Copilot button on the toolbar .” Then, back in the Copilot and sidebar settings, turn off the “Show sidebar button” toggle that has just appeared.
Click Languages in the left-hand navigation. Disable “Use Copilot for writing on the web.”
The latest Firefox (148 and later) has a feature to block all AI enhancements. So update, if you can, go to Settings and look for AI Controls . Turn Block AI Enhancements on (or select more granularly from the list)
Using Firefox, this extension removes a lot of the AI junk if you use Google as your default search engine.
DDG is both a browser and a search engine. They have a no-AI version you can get to by going to https://noai.duckduckgo.com / You can also change your default search engine to this and stop using Google, here are steps for every browser . If you want to go all out and get the DDG browser, they also have instructions on how to do that .
In Gmail, click the Settings (gear) icon, and then select See all settings . On the General tab, scroll down to Google Workspace smart features . Click Manage Workspace smart feature settings and toggle off two options: Smart features in Google Workspace and Smart features in other Google products . Uncheck the box next to Turn on smart features in Gmail, Chat, and Meet on the same tab.
In Gmail for Android the workspace smart features are under the Gmail hamburger/dropdown menu – Settings – Account name
Owners and admins of Slack workspaces have some options in disabling AI features. Here is their help page about managing those features .
AI lives in here in a few places. Suggested Replies, go to Settings – Chats – Suggestions & smart replies and toggle off Suggested replies . AI Sticker suggestions can be shut off in that same menu. For AI message summaries, those are managed in a different location: Settings – Notifications – AI message summaries . Depending on what operating system you’re running, it may also be in Settings – Chat – Private Processing – Private Processing Features
Copilot exists both as an embedded AI agent in Office 365 and also built into the Windows 11 operating system. Right-click the Copilot entry in the Start menu and select Uninstall . If that option isn’t there, head over to your installed apps list ( Start – Settings – Apps ) and uninstall Copilot from there.
In certain builds of Windows 11, Copilot is stuck into the OS, so a simple uninstall might not work. In that case, you can toggle it off via the settings: Start – Settings – Personalization – Taskbar – turn off Copilot File Explorer has some AI built into it which can be turn-onable accidentally. To remove. Start – Settings – Privacy & Security – Click to Do.
Notepad has its own special Copilot, so you’ll need to disable AI there separately. Open the Notepad settings, find the AI features section (or a sparkly icon near a Writing Tools section), and toggle Copilot off.
Office 365/Outlook There is a separate Enable Copilot checkbox in each app and the checkbox only applies to that app on that device. Read more about how to turn it off . If you don’t have that option, you can change your privacy settings to disable Copilot .
“ Text and image generation ” are turned on by default, affecting Notepad, Photos, Snipping Tool, and Xbox. Turn this off. It is complicated, though not impossible, to remove Copilot from your Windows computer entirely. Here is a link that gets into the details . If you feel comfortable installing software O&O ShutUp 10 is one way to do that, Win11DeBloat is another.
Cick the … More on the bottom of the left-hand sidebar, select Settings, select AI Features and switch Message summaries off.
Sign in at Zoom.com. Scroll to My Account on the left and select Settings . Go to the Zoom AI tab on the right where you can disable everything. Note: new AI options get added and they default to “on” so check back. To the right of the Zoom AI tab click on My Notes and disable “Allow participants to transcribe meetings with My Notes”
Inspired by
the librarians from Bangor
.
Last updated 16aug26
Github has been down consistently over the last few months - does it make sense to switch to alternatives now?
When do you think majority will switch?
The conventional thinking for Democrats hoping to win in Florida is to be tough on Cuba, friendly to Israel, and careful to avoid appearing too liberal. It’s a tactic aimed at courting powerful voting blocs, and one that helped Rep. Jared Moskowitz take office in the land of sunny skies and low taxes.
This year, however, he’s facing a challenger attempting to flip that strategy on its head. A Democratic Socialists of America member, Oliver Larkin, is trying to win a primary against Moskowitz by running far to his left on both foreign and domestic policies.
The pair presents a striking contrast: Moskowitz has drawn heavy support from donors associated with the American Israel Public Affairs Committee and an artificial intelligence super PAC, while Larkin, a 34-year-old union organizer, calls himself an anti-Zionist and leans on small-dollar funders.
Polls of the race have produced vastly different results, with one commissioned by Larkin putting him within striking distance, while another from Moskowitz’s campaign showing the incumbent far ahead.
Susan McManus, a professor emeritus of political science at the University of South Florida, said in an email that the race will be a significant tell of the mood in the Democratic electorate.
“Of all the Florida congressional districts, this one will be the biggest test of how deep is the generational and ideological divide in the Florida Democratic Party, and to what extent the socialism label may splinter the Latino vote,” she said.
That label is one that Larkin wears proudly, describing himself on his website as an active member of the Broward County Democratic Socialists of America. He supports Medicare for All while opposing the embargo on Cuba and weapons transfers to Israel.
In an interview with The Intercept ahead of Tuesday’s primary, Larkin said he rejected the Florida Democratic Party’s tendency to lean right to win Republican and independent votes. That strategy has not worked, Larkin said, pointing to Republican-turned-Democrat Charlie Crist’s big loss to Gov. Ron DeSantis in 2024 .
Instead, Larkin argues that an openly leftist candidate can energize the Democratic base while winning over disaffected independents.
“I think another false choice that corporate Democrats make is that non-party affiliated or independent voters want to see candidates split the difference right down the middle. I don’t think that’s true,” he said. “They see the status quo politics as completely broken.”
Moskowitz’s campaign did not respond to an interview request. On his campaign website, however, he leans heavily on his bipartisan bona fides, noting that he served in DeSantis’s cabinet as emergency management director until his election to Congress in 2022.
That credential could help win over voters in a redrawn district spanning much of the coast near Fort Lauderdale that would have voted for President Donald Trump in the 2024 election. Moskowitz already represents about half of the new district’s voters in his current 23rd Congressional District.
The new 25th District also has one of the nation’s largest Jewish populations, making Moskowitz’s outspoken support for Israel particularly salient. Last week, pro-Israel activists in south Florida pressured multiple venues to cancel a joint rally that Larkin was to hold with Rep. Rashida Tlaib , one of the Democratic Party’s most outspoken critics of Israel.
Moskowitz has also accused Larkin — without offering any evidence — of running against him based on his religion, in a letter to the Sun Sentinel where he declined to participate in a joint interview with Larkin.
The South Florida newspaper called that charge “demonstrably untrue” and dinged Moskowitz for dodging opportunities to face off against his opponent.
“The Sun Sentinel has endorsed Moskowitz in all of his previous races. But in this case, his continued avoidance of Larkin shows a lack of respect for voters and can be viewed as an unwillingness to defend his own record,” the paper’s editorial board said.
Larkin was born in Florida, but after his stint as a volunteer and staffer for Sen. Bernie Sanders, he worked for a consulting firm aligned with the Democratic Party in Washington, D.C. He returned to the Sunshine State in 2022. During his run for office this cycle, he has posted respectable fundraising numbers for a first-time candidate taking on an incumbent. In an August 13 press release, his campaign said he had raised over $1 million.
Still, that puts him far behind Moskowitz, who reported receipts of nearly $3 million during this election cycle in his most recent campaign finance report, with $1.8 million remaining on hand.
While some pro-Israel candidates have sought to avoid association with AIPAC this year, given the group’s increasingly toxic brand, Moskowitz has defended its donations to his campaign. His fundraising reports show that he has raised $953,000 from donors associated with AIPAC, according to the transparency website Open Secrets .
An artificial intelligence super PAC is also spending heavily on Moskowitz. The super PAC, Leading the Future, has spent more than $300,000 thus far.
“He doesn’t have a volunteer base, because nobody wants to volunteer for him.”
Larkin believes he can make up the divide in spending on advertising with old-fashioned door-knocking. His campaign boasts of having made more than 200,000 voter contacts — and he says there has been little evidence that Moskowitz’s campaign is attempting to match their ground game.
“He’s got ads on TV. He’s got mail. He has no ground game to speak of. I don’t believe he is even attempting to knock on doors,” Larkin said. “He doesn’t have a volunteer base, because nobody wants to volunteer for him.”
Version 5.0 of GNU Poke , a binary-data editor, has been released. This release includes a number of improvements to the Poke compiler, additions to the Poke language, as well as runtime and standard library updates. See below for the full list of changes.
| From : | Mohammad-Reza Nabipoor <mnabipoor-AT-gnu.org> | |
| To : | info-gnu-AT-gnu.org | |
| Subject : | GNU poke 5.0 released | |
| Date : | Mon, 17 Aug 2026 10:41:51 +0200 | |
| Message-ID : | <aoLI_baPd-SOP2D8@feynman> | |
| Cc : | "Jose E. Marchesi" <jemarch-AT-gnu.org> |
I am happy to announce a new major release of GNU poke, version 5.0. GNU poke 5.0 release is now available at https://ftp.gnu.org/gnu/poke/poke-5.0.tar.gz The tarball is signed and you can get the PGP signature at https://ftp.gnu.org/gnu/poke/poke-5.0.tar.gz.sig GNU poke (http://www.jemarch.net/poke) is an interactive, extensible editor for binary data. Not limited to editing basic entities such as bits and bytes, it provides a full-fledged procedural, interactive programming language designed to describe data structures and to operate on them. I'd like to thank everyone who contributed to this release through code, documentation, or testing. What is new in this release: * User interface updates - Now hyperlink server can bind to a user-specified port for listening to commands (-p, --hserver-port). * Poke Language updates - Floating-point arithmetic is now supported on uint<32>/uint<64> types. uint<32> will be interpreted as a single-precision floating-point number and uint<64> will be interpreted as a double-precision floating-point number as defined per the IEEE 754 standard. The following expressions are now supported: - Addition: a .+ b - Subtraction: a .- b - Multiplication: a .* b - Division: a ./ b - Ceil-devision: a ./^ b - Exponentiation: a .** b - Remainder: a .% b - Post-increment: a.++ - Pre-increment: .++a - Post-decrement: a.-- - Pre-decrement: .--a - Negation: .-a - Less-than: a .< b - Less-than-or-equal-to: a .<= b - Greater-than: a .> b - Greater-than-or-equal-to: a .>= b - Equal-to: a .== b - Not-equal-to: a .!= b * Poke Runtime updates - Thanks to the great work of David Faust, poke now supports reactive IO spaces! Extent of a PVM value mapped in a given IO space will be tracked and values will be re-mapped only if a write happens in their extent; which is a big performance win for read-intense programs. - A bunch of undefined behavior (UB) releated to left-shifts has been fixed. - Improved human-readable message of E_conv exception when verifying length/size of an array with dynamic bound(s) to help the user to understand the mistake. * Poke compiler updates - Now poke can properly handle writes to nested integral struct/unions fields. Previously write to nested fields of integral structs did not materialize in IO space. * Standard Poke Library updates - Closure's pretty printer now adds closure's name (identifier) to output. - Two new functions to calculate square root of single and double precision floating point numbers: sqrtf and sqrtd. They accept uint<32> and uint<64> respectively as the IEEE 754 single and double precision floating-point numbers. * libpoke updates - Version of DSO is bumped to 2.0.0, and from this release onward, we try to not break the ABI, and bump the version components according to the libtool's recommendation (when needed). - To be able to keep the ABI backward-compatibility promise, all public APIs are now accepting either pk_compiler or pk_val. This is the first step toward removing global state from libpoke to be able to have multiple instances of libpoke in a single process (and also to be able to accomplish thread-safety). We're not there yet, but we'll be there some day (hopefully soon)! * IO subsystem updates - IOS_F_TRUNCATE has been re-introduced (it was removed after release of poke 1.0 by the rationale that it's not that useful of a flag. Turns out it's quite useful to start from an empty file when assembling binary files from scratch using poke. * Pickles updates - Improved ustar pickle and add tests. Method get_last_mod_time has been fixed and the following methods has been added: get_{file,owner_user,group}_name. - Improved time pickle to print date and time properly (zero-padded), and also add ptime_str function to get date/time information as a string. * Platform supports - We tried to improve MinGW compilation situation by importing more Gnulib modules, but we still cannot have poke executable for MinGW platform. Help is very much appreciated in this area! * Documentation updates - Thanks to people who actually read the reference manual, this release includes a bunch of corrections to the documentation! Cheers to them! Happy poking! Mohammad-Reza Nabipoor
Author: Len Noe, Solutions Architect, BeyondTrust
Every mature Active Directory environment has a component that quietly holds more power than the people running it usually admit: the Certification Authority (CA). The thing your entire estate has agreed to believe.
When it signs a certificate, every machine, service, and authentication flow downstream treats that signature as truth. That is an enormous amount of trust concentrated in one system, and most organizations manage it like a utility installed once and never thought about again.
Certighost , tracked as CVE-2026-54121 , is a reminder of what happens when that trust is misplaced. Researchers published a working proof-of-concept on July 24, 2026, demonstrating that a low-privileged Active Directory user (holding nothing more than a standard domain account) can coerce an Enterprise CA into issuing a valid authentication certificate for a Domain Controller, then use that certificate to become the Domain Controller.
Microsoft shipped the fix on July 14, 2026, and rated it 8.8 on the CVSS scale.
Active Directory Certificate Services is Microsoft's public key infrastructure, issuing and managing the certificates that underpin smart card logon, device and user authentication, and VPN access. A standard domain user has no business obtaining a certificate that represents a Domain Controller, yet Certighost breaks that boundary without touching a single access control list.
The flaw lives in an AD CS enrollment behavior known as "chase" functionality. When an Enterprise CA cannot immediately resolve the target object locally, it can follow requester-supplied routing information (a parameter called cdc) to look the object up elsewhere.
The defect is that the CA never verifies that the endpoint named in cdc is a legitimate Domain Controller before it reaches out to it. An attacker points cdc at a machine they control and the CA dutifully makes an outbound connection to that rogue endpoint, which answers with forged identity data, including the target Domain Controller’s object security identifier and DNS host name.
The CA trusts what it is told, binds that identity to a signed X.509 certificate, and hands the attacker a certificate that says they are a Domain Controller.
From there, the attack follows a well-understood path. The attacker uses the certificate with PKINIT, the public key extension to Kerberos, to obtain a Ticket Granting Ticket as the Domain Controller's machine account.
Domain Controller accounts inherently hold directory replication rights, enough to run a DCSync operation against a real DC and pull credential material, up to and including the krbtgt account hash. Once you have krbtgt, you can forge Kerberos tickets at will, and the domain is functionally yours.
A standard Domain User account was sufficient in testing because default Active Directory settings, including the default MachineAccountQuota that lets ordinary users create machine accounts, provided everything the chain needed.
As of public disclosure, there was no confirmed exploitation in the wild. That is not a reason to relax. A functional, public proof-of-concept collapses the effort required to reproduce this, and the gap between "PoC exists" and "commodity tooling includes it" is measured in weeks, not years.
Certighost exposed how privilege buried in trusted relationships and overlooked defaults can become a path to domain compromise.
BeyondTrust’s complimentary Identity Security Risk Assessment helps you uncover those hidden identity and privilege exposures across your own environment before they become the next path attackers exploit.
It is tempting to file Certighost under PKI arcana, assign it to whoever owns the CA, and move on once the patch lands.
Strip away the certificate machinery and look at the shape of the attack: an unprivileged identity manipulated a trusted system into vouching for a privileged identity, and the environment had no mechanism to question the result. That is a trust-validation problem that sits at the core of identity security.
The Certification Authority is not a passive appliance. It is a privileged identity in its own right, one that manufactures trust on behalf of the entire domain. The patch Microsoft shipped is, at its heart, a verification step enforcing that the target of a chase lookup is genuinely a Domain Controller.
That is the recurring signature of identity-driven compromise: the attacker rarely breaks cryptography or authentication. They find the place where the system decided to trust without checking.
There is a second, more uncomfortable lesson buried in the prerequisites. The default configuration of Active Directory grants every authenticated user a small piece of standing privilege: the ability to create machine accounts, courtesy of a MachineAccountQuota that permits it by default.
Certighost is one of many attack chains that quietly depend on that standing capability. The specific CVE is new, but the latent privilege it leaned on has been sitting in your domain for years.
The vulnerability created a shortcut, but the terrain was already dangerous.
A determined attacker who lands a single low-privileged foothold has a realistic path to domain dominance because privilege has accumulated in places no one is actively governing: overbroad certificate template permissions, permissive machine account defaults, flat trust between the CA and the domain, and monitoring that watches endpoints but not the identity control plane.
Certighost is a clean demonstration of how those conditions compound. Remove the CVE and the underlying exposure remains, waiting for the next technique.
Patch first. Apply Microsoft's July 14, 2026 update to every issuing Certification Authority because it introduces the destination validation that shuts down the specific chase abuse.
If deployment is delayed, researchers documented a workaround that disables the vulnerable chase functionality. But test it before you deploy it: that path exists to support legitimate enrollment workflows, and turning it off can break them.
Beyond the immediate fix, reduce the standing privilege the attack relied on. Setting the domain's MachineAccountQuota to zero removes the default ability for ordinary users to create machine accounts, meaningfully shrinking the attack surface for this class of technique.
That change is not free. Some provisioning workflows and legacy tooling assume users can join machines to the domain, so inventory those dependencies and route machine creation through controlled, delegated accounts rather than leaving it open to everyone.
Then constrain the CA itself. Restrict outbound SMB and LDAP from your Certification Authorities so they can only communicate with known, authorized Domain Controllers, which directly undercuts the rogue-endpoint step in the chain.
Review Enterprise CA deployments, certificate templates, and enrollment permissions: which principals can request this, and does that population have any business holding the identity this certificate represents?
Most environments have never audited certificate enrollment rights against that standard, and that is precisely where AD CS attack paths originate.
Finally, watch the right layer. Monitor for anomalous machine account creation, unusual certificate enrollment activity, and DCSync operations, and pay attention to CA enrollment events rather than assuming endpoint telemetry will catch an identity attack it was never designed to see.
DCSync from anything other than a Domain Controller deserves an immediate response, and if your detection stack cannot surface it, that is a gap worth closing.
Certighost will be patched, cataloged, and largely forgotten within a quarter. That is the trap. If the response stops at the KB number, the organization learns nothing durable because the specific bug was never the point. The point is that trust in an enterprise is a thing you architect and continuously validate, not a property you configure once and inherit forever.
The defensible posture is not a longer patch list. It is a mindset that treats identity as infrastructure and privilege as risk to be minimized rather than convenience to be preserved. Reduce standing privilege wherever it hides, including the defaults you never chose, and validate trust at every point where a system is about to act on it, not just at the front door.
Your Certification Authority has been handing out trusted identities on your behalf since the day it was stood up. The work is making sure it only does so for identities you can actually verify.
Learn how BeyondTrust’s complimentary Identity Security Risk Assessment helps you uncover those hidden identity and privilege exposures across your own environment.
Len Noe is a Solutions Architect at BeyondTrust, Transhuman, Podcaster, International Cyber Security Speaker, Author, Technical Evangelist, and Biohacker with 13 implanted microchips.
A former blackhat with more than 30 years in technology, he has presented in over 70 countries and is featured in the documentary I Am Machine, which premiered at DEF CON 2025.
BeyondTrust is the global leader in privilege-centric identity security protecting Paths to Privilege™. Identity alone doesn’t create risk. Privilege does. As human, machine, and AI agent identities explode across every environment, BeyondTrust is the only company built to discover, control, and secure privilege across all of them from a single platform. Trusted by 20,000+ customers, including 75 of the Fortune 100, and recognized as a multi-category leader by top industry analysts, BeyondTrust reframes identity security from a management problem into a strategic advantage.
Learn more at www.beyondtrust.com .
Sponsored and written by BeyondTrust .
|
|
I've used gitlab and gitea; gitea is faster, and easier to manage and does everything I actually need though, is less feature complete. |
|
|
|
I strongly disagree with the assumption that GitHub's alternative is another centralized forge. Git itself is perfectly decentralized, as was the original Linux kernel development process. How people managed to put all their eggs in one intermittently available service is beyond me. Moving the eggs into another bucket is not a solution (like Microsoft is short of servers). The SPoF is the problem. There are plumbing, porcelain and "github" layers. The "github" part has to be decentralized as well. Then, using a particular forge will be a choice of convenience, not necessity. https://replicated.live/blog/crdt |
|
|
|
I use Forgejo + Gitea for my home forge, then Tangled for anything I want to share with my own runner. I self host Lore for my gamedev projects. |
|
|
|
I'm mostly using GitLab right now, both self-hosted and their hosted platform. But am curious about Cursor Origin and certainly plan to try that out when it's available. |
|
|
|
I personally host a forgejo instance on a private VPS ; so far almost no maintenance except protecting it from ai-crawlers[#1]. If you don't want the hassle, codeberg.org is a public instance of forgejo. [#1]: https://her.esy.fun/posts/0031-how-i-protect-my-forgejo-inst... I configure my local repositories to push on both Github and my forgejo instance. I am not using the CI much for my private projects (local tests are enough in my case). |
|
|
|
My org uses a self-hosted instance of RhodeCode Enterprise. It's not as feature rich as GitHub, but it's worked well for us. |
|
|
|
My org's GitHub Enterprise never goes down. The feature set is almost the same, though it lags a few months behind. At least you don't have to learn anything new. |
|
|
|
If an org is heavily invested in GitHub Actions and GitHub App integrations, is self-hosting GitHub enterprise the only practical option? |
|
|
|
Buildkite has built a GitHub Actions adapter... a good first step out of the GitHub Actions supply chain attack trap. |
|
|
|
My org self hosts the community version of gitlab and we are perfectly happy with it. Manage your own infrastructure, put the work into maintaining it and you'll have much fewer headaches. |
|
|
|
my favourites are sourcehut (that has excellent ci, and does not try to be a github clone) and codeberg (with slightly more straightforward migration path from github) [0] sr.ht [1] codeberg.org |
|
|
|
Does sourcehut still require patches via email instead of "pull requests"? That was the deal breaker for me last time I looked at it. |
|
|
|
We're moving over to self-hosted forgejo. Interface is roughly similar, featureset is roughly similar enough. |
August 14, 2026 • Tarun Pothulapati (Engineer)
Every day, turbopuffer customers ask for things: new query plans, new APIs, new index structures. In response, we deploy dozens of database upgrades per day across our clusters, many of them the same day we open the PR. Shipping fast is how we make every customer feel like they're our only customer .
We don't want to limit where you can run turbopuffer, so we support many regions across three deployment models: public SaaS , single-tenant SaaS , and BYOC . In total, we operate 100+ clusters, twice as many as we had 6 months ago, and growing as we add more public regions and many more BYOC deployments.
╔═ turbopuffer cloud account ═════════════════════╗ ╔═ customer cloud account ═══╗
║ ║░ ║ ║░
║ ┏━ public ━━━━━━━━━━┓ ┏━ single-tenant ━━━━━┓ ║░ ║ ┏━ BYOC ━━━━━━━━━━━━━━━┓ ║░
║ ┃ AWS | GCP ┃ ┃ AWS | GCP ┃ ║░ ║ ┃ AWS | GCP | Azure ┃ ║░
║ ┃ shared resources ┃ ┃ dedicated resources ┃ ║░ ║ ┃ customer's resources ┃ ║░
║ ┃ ┃ ┃ ┃ ║░ ║ ┃ ┃ ║░
║ ┃ tpuf operator ┃ ┃ tpuf operator ┃ ║░ ║ ┃ no tpuf operator ┃ ║░
║ ┃ access ┃ ┃ access ┃ ║░ ║ ┃ access ┃ ║░
║ ┗━━━━━━━━━━━━━━━━━━━┛ ┗━━━━━━━━━━━━━━━━━━━━━┛ ║░ ║ ┗━━━━━━━━━━━━━━━━━━━━━━┛ ║░
║ ║░ ║ ║░
╚═════════════════════════════════════════════════╝░ ╚════════════════════════════╝░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░╔═ tpuf account ═══════════╗
║ ┏ public ━━━━━━━━━━━━━━┓ ║
║ ┃ AWS | GCP ┃ ║
║ ┃ shared resources ┃ ║
║ ┃ tpuf operator access ┃ ║
║ ┗━━━━━━━━━━━━━━━━━━━━━━┛ ║
║ ┏ single-tenant ━━━━━━━┓ ║
║ ┃ AWS | GCP ┃ ║
║ ┃ dedicated resources ┃ ║
║ ┃ tpuf operator access ┃ ║
║ ┗━━━━━━━━━━━━━━━━━━━━━━┛ ║
╚══════════════════════════╝
╔═ customer account ═══════╗
║ ┏ BYOC ━━━━━━━━━━━━━━━━┓ ║░
║ ┃ AWS|GCP|Azure ┃ ║░
║ ┃ customer's resources ┃ ║░
║ ┃ no tpuf operator ┃ ║░
║ ┃ access ┃ ║░
║ ┗━━━━━━━━━━━━━━━━━━━━━━┛ ║░
╚══════════════════════════╝░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░
BYOC clusters live inside our customers' cloud accounts, to which we hold no
credentials by default. We can't just SSH or
kubectl
in. Some BYOC vendors
solve this by asking the customer to carve out a dedicated cloud account and
grant the vendor standing admin credentials inside it. That keeps the rest of
the customer's cloud account isolated, but every dedicated account creates
security monitoring + compliance + billing overhead that we generally prefer to
avoid.
We don't want different control planes for BYOC and SaaS, so we have to design for the lowest common denominator. We must be able to operate every cluster without reaching in.
The only way this works is if every operation we need to perform on a cluster can run without us reaching in. The cluster must be able to independently drive its operations to a terminal state, even if it loses its connection to the central control plane.
The solution to this is standard Kubernetes stuff. On every cluster, we run a
local cluster agent that implements a simple state machine. A single Kubernetes
CRD
called
TurbopufferOperation
expresses every operation kind, from
upgrade
to
tidy
. A Kubernetes
controller drives each custom resource (CR) from state to state via a
reconciliation loop until it reaches a terminal state. Operations advance on
their own by default, but BYOC customers can gate operations on
approval
or
maintenance windows
.
These are baked into the CRD as waiting states that advance when approvals are
given or the window opens.
╔═ operation lifecycle ═══════════════════════════════════════════╗
║ ║░
║ ┏━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ REQUIRES_APPROVAL ┃ ║░
║ ┗━━━━━━━━┯━━━━━━━━━━┛ ║░
║ │ approve (auto or manual) ║░
║ ▼ ║░
║ ┏━━━━━━━━━━━━━━━━━━━┓ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ PENDING ┃ ──▶ ┃ AWAITING_MAINTENANCE_WINDOW ┃ ║░
║ ┗━━━━━━━━┯━━━━━━━━━━┛ ◀── ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ ║░
║ │ start() ║░
║ ▼ ║░
║ ┏━━━━━━━━━━━━━━━━━━━┓ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ RUNNING ┃ ──▶ ┃ AWAITING_EXTERNAL_EXECUTION ┃ ║░
║ ┗━━━━━━━━┯━━━━━━━━━━┛ ◀── ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ ║░
║ │ poll() ║░
║ ┌────┴────┐ ║░
║ ▼ ▼ ║░
║ ┏━━━━━━━┓ ┏━━━━━━━┓ ║░
║ ┃SUCCESS┃ ┃FAILURE┃ ║░
║ ┗━━━━━━━┛ ┗━━━━━━━┛ ║░
║ ║░
╚═════════════════════════════════════════════════════════════════╝░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░╔═ operation lifecycle ════╗
║ ┏━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ REQUIRES_APPROVAL ┃ ║░
║ ┗━━━━━━━━━━━┯━━━━━━━━━━┛ ║░
║ ▼ approve ║░
║ ┏━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ PENDING ┃ ║░
║ ┗━━━━━━━━━━━┯━━━━━━━━━━┛ ║░
║ ▼ start() ║░
║ ┏━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ RUNNING ┃ ║░
║ ┗━━━━┯━━━━━━━━━━━┯━━━━━┛ ║░
║ ▼ poll() ▼ ║░
║ ┏━━━━━━━━━┓ ┏━━━━━━━━━┓ ║░
║ ┃ SUCCESS ┃ ┃ FAILURE ┃ ║░
║ ┗━━━━━━━━━┛ ┗━━━━━━━━━┛ ║░
╚══════════════════════════╝░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░
The key is to define states that are generic enough to model every operation
kind, both existing and future, but finite enough that the reconciler can always
drive them toward a terminal state. We never need to reach in to do work, and
the state lives as a durable object in the cluster's own
etcd
. If the agent crashes
or loses its connection to the control plane, it can still drive that work to
completion.
The local, durable state machine means we don't have to reach into a cluster to drive its work, but how does a cluster get its work?
The default here would be to reach for infrastructure-as-code (IaC) tools like Terraform and Helm. These tools are great for provisioning infrastructure. We use them for that! But they're the wrong control plane for operating a fleet of databases.
First, we do not have a dedicated DevOps function at turbopuffer. We ask our database engineers to deploy their code to the infra. Most of them haven't used IaC tools much in the past, so it doesn't make sense to put those tools in their critical path.
Second, Terraform's default model would require us to run
terraform apply
from
our own infrastructure with the customer's infra as the target, which we can't
do for BYOC. A GitOps flow for Terraform could solve this by having the customer
automatically trigger
terraform apply
from within their cloud account whenever
a Git repo is updated. For database upgrades, this could theoretically work:
just bump the Docker image in the Terraform config. But who owns the repo? If
it's ours, BYOC customers who want an approval gate have no control over merges.
If it's theirs, we're back to needing write access into their systems or waiting
on humans to merge the PR.
Either way, many of our operations have a less declarative shape. Things like one-off namespace reindexing, compacting a WAL , or garbage collecting the LSM are jobs to be done, not states to be arrived at. Every ad hoc operation would need to be a git commit with a new manifest to create the job, then a commit to garbage collect the completed jobs. Terraform files make great declarative manifests, but terrible job queues.
We can't reach into a cluster, and we won't use GitOps, but the cluster still needs to be able to pull its work from the control plane and push its status back out. The system must be fully tolerant of a lost connection between cluster and control plane; a control plane outage should not take down an otherwise healthy cluster. In the event of a lost connection, the cluster should be able to pick up its new operations once it reconnects, while the control plane should be able to catch up on the state of the cluster.
To accomplish these aims, we built a custom central control plane consisting of an API server backed by a PlanetScale MySQL database.
Each cluster agent is given a cluster-specific API key. To fetch its operations,
the agent regularly polls the API server via an authenticated
GET
request, to
which the API server responds with that cluster's pending operations. Once the
agent gets its operations, it stores them as custom resources and starts running
them through its reconciliation loop. This is idempotent: each operation's CR is
named with that operation's ID, so if for some reason a later
GET
returns the
same operation again, the agent will just find the existing CR and resume from
the state stored in
etcd
instead of creating a duplicate.
As the operations advance, the cluster agent will periodically
POST
its
buffered status transitions back to the API server, which mirrors them into a
status_transitions
table in MySQL as an append-only log.
tpuf engineers
│
▼
╔═ control plane ═══════╗ ╔═ cluster ══════════════════╗
║ ║░ ║ ║░
║ ┏━━━━━━━━━━━━━━━━━━━┓ ║░ ║ ┏━ cluster agent ━━━━━━━━┓ ║░
║ ┃ UI ┃ ║░ ║ ┃┌──────────────────────┐┃ ║░
║ ┗━━━━━━━━━┯━━━━━━━━━┛ ║░ ║ ┃│ k8s controller │┃ ║░
║ ▼ ║░ ║ ┃│ (sync + reconciler) │┃ ║░
║ ┏━━━━━━━━━━━━━━━━━━━┓ ║◀─── GET operations ───║ ┃└───────────┬──────────┘┃ ║░
║ ┃ API server ┃ ║░ ║ ┃ ▼ ┃ ║░
║ ┗━━━━━━━━━┯━━━━━━━━━┛ ║◀── POST transitions ──║ ┃┌──────────────────────┐┃ ║░
║ ▼ ║░ ║ ┃│ TurbopufferOperation │┃ ║░
║ ┏━━━━━━━━━━━━━━━━━━━┓ ║░ ║ ┃│ CRs │┃ ║░
║ ┃MySQL (PlanetScale)┃ ║░ ║ ┃└──────────────────────┘┃ ║░
║ ┗━━━━━━━━━━━━━━━━━━━┛ ║░ ║ ┗━━━━━━━━━━━━━━━━━━━━━━━━┛ ║░
║ ║░ ║ ║░
╚═══════════════════════╝░ ╚════════════════════════════╝░
░░░░░░░░░░░░░░░░░░░░░░░░░ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ tpuf engineers
│
▼
╔═ control plane ════════════╗
║ ┏━━━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ UI ┃ ║░
║ ┗━━━━━━━━━━━┯━━━━━━━━━━━━┛ ║░
║ ▼ ║░
║ ┏━━━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ API server ┃ ║░
║ ┗━━━━━━━━━━━┯━━━━━━━━━━━━┛ ║░
║ ▼ ║░
║ ┏━━━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ MySQL ┃ ║░
║ ┃ (PlanetScale) ┃ ║░
║ ┗━━━━━━━━━━━━━━━━━━━━━━━━┛ ║░
╚════════════════════════════╝░
░░░░░▲░░░░░░░░░░░▲░░░░░░░░░░░░
│GET work │POST state
│ │
╔═ cluster ══════════════════╗
║ ┏━ cluster agent ━━━━━━━━┓ ║░
║ ┃┌──────────────────────┐┃ ║░
║ ┃│ k8s controller │┃ ║░
║ ┃│ (sync + reconciler) │┃ ║░
║ ┃└───────────┬──────────┘┃ ║░
║ ┃ ▼ ┃ ║░
║ ┃┌──────────────────────┐┃ ║░
║ ┃│ TurbopufferOperation │┃ ║░
║ ┃│ CRs │┃ ║░
║ ┃└──────────────────────┘┃ ║░
║ ┗━━━━━━━━━━━━━━━━━━━━━━━━┛ ║░
╚════════════════════════════╝░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
If the agent's
POST
fails to get a response, the agent puts the transitions
back into its buffer and tries again. The
POST
may have landed even if the
response didn't come back, so the same transition could show up twice in the
log. That doesn't matter: an operation's current state is just its latest
transition.
This is how we keep the central control plane in sync with each cluster even in the event of temporary connection loss. The database maintains the source of truth for the desired operations we want to run on every cluster, and holds a mirror of the status of those operations so we can rebuild a cluster's state from afar.
But a database is a dangerous interface. We said we didn't want to ship code
with YAML — it would be insane to replace that with
INSERT INTO
SQL
statements.
We deploy every day, multiple times per day, and debug and maintain the clusters in between deploys. If you're a turbopuffer database engineer, you want the control plane to be as reliable and easy to use as your favorite code editor.
The control plane has a custom Remix/React dashboard that fronts the API server. The UX is inspired by Linear: everything has a hotkey, so you can operate it entirely from the keyboard. The information hierarchy is structured, and the selection state is clear, so you know exactly which clusters you're working on.
Upgrades are the most common operation, and the UI reflects that. We can quickly see each cluster's deployed commit SHA and how far it has drifted from the latest passing SHA, with metadata pulled from GitHub (commit message, PR number, commit date) so we don't have to decipher the SHA. Upgrades are typically done in groups, so we can select multiple clusters across all deployment models, choose a target commit SHA, and upgrade all of them from the keyboard. We can similarly kick off all other operations from this UI.
0:00 / 0:00
We don't expect our engineers to be in the dashboard all the time, so we stream status transitions for each operation into a dedicated Slack thread. If an operation fails, it fails loudly into the channel for people to react to and fix. This gets our attention without us having to babysit.
When an approval is needed for operations on a BYOC cluster, the same Slack integration automatically notifies the customer through their dedicated Slack support channel so they can approve.
01 / 02
The control plane also provides a convenient jumping-off point for all the debugging and monitoring interfaces we use. If we want to watch an operation closely, we can jump from the control plane directly into Datadog logs, traces, or metrics scoped to its cluster(s). When a customer reports high latency, we can quickly pull up the memory/CPU profile in Polar Signals for their query pods. When a customer wants a limit bumped for a namespace, we can inspect the namespace and update the config right there (really!). When all the tools you need are right at hand, you feel more connected to your infra, and on-call sucks a lot less.
0:00 / 0:00
This dashboard is not just a vibe-coded sidecar to the control plane. We treat it as first-class software and invest engineering effort into it, so the complexity of operating the clusters stays mostly hidden behind a simple, fast, keyboard-driven interface.
While the control plane has scaled very well, we started to feel the toil of fleet-wide upgrades: manually selecting clusters and a target SHA, kicking off the upgrade, monitoring it, and making sure things looked ok before proceeding to the rest. The larger our fleet got, the more painful it became to sequence and babysit a rollout that could last hours.
Now, any operation can be submitted as a fleet operation, and the control plane's fleet controller will roll it out across a sequence of waves containing one or more clusters. Between waves, a gate checks that every cluster in the wave upgraded successfully and that our monitors are quiet. If a monitor fires or any cluster fails, the fleet controller pauses the rollout and notifies us in Slack, limiting the fallout from a bad deploy.
Importantly, nothing changes for the cluster agent. It has no concept of a fleet or fleet operation. It still polls the API server for its pending operations, and its status transitions are still mirrored back into the same MySQL table. Every operation is still the same CRD driven by the same local state machine. Orchestrating 100s of them is effectively just a loop over existing primitives.
turbopuffer's architecture is easy to reason about and easy to be on call for. It would be a shame if it wasn't also easy to deploy to. The design decisions we make in the control plane all funnel toward simplicity: a single control plane for all deployment models, a single CRD for every operation, a single UI to manage the entire fleet.
This simplicity has allowed us to scale the control plane from managing just a few public regions to 100+ clusters across dozens of regions and multiple deployment models. We believe its simplicity will allow us to maintain our shipping speed even as we scale to 1000s.
turbopuffer is a fast search engine that hosts 1T+ documents, handles 10M+ writes/s , and serves 25k+ queries/s . We are ready for far more. We hope you'll trust us with your queries.
Today we are releasing BrowserPod 3.0, with full support for running any Rust application in the browser, many fixes for Node.js, and initial Python support.
Our Rust support goes beyond what can normally be achieved with the existing Wasm targets in terms of standard library features and third-party crate compatibility. Programs can access the filesystem, make network requests, run subprocesses, and interact with concurrently running applications. All of this without changing a single line of code !
In the demo below, you can see the preview version of
Yarn 6
(written in Rust) installing an NPM project:
BrowserPod is an in-browser code sandbox. Its goal is to make it possible to run any Linux application in modern browsers by compiling the application to WebAssembly and providing the full Linux syscall interface.
It provides an efficient, locally persistent virtual filesystem, outbound internet access for downloading packages or calling APIs, and inbound connections for exposing local development servers. BrowserPod supports real parallelism by running each thread or process on an independent Worker, while providing a consistent view of the system to all the running programs. For all purposes, it can be considered an OS kernel for the Web platform , implemented in WebAssembly.
This set of features makes BrowserPod uniquely suited for safe in-browser agentic code execution, web-based IDEs and development environments, interactive docs and live demos, educational platforms, and other applications that benefit from sandboxed execution inside a web app.
BrowserPod’s ambition is to run an entire Linux userspace in the browser. For the most part, this used to mean compiling a bunch of C/C++ projects.
Most dynamic languages, such as Python, JavaScript or Ruby run on top of interpreters and runtimes that are also, most usually, written in C/C++, and so having a C/C++ toolchain (in our case,
Cheerp
) would get you very far.
Nowadays, this is less and less true. Popular languages like Rust and Go are compiled ahead of time like C/C++, and have their own toolchains.
Many build tools for Node.js in particular are being written (or rewritten) in Rust.
For these reasons, Rust was always part of our roadmap for BrowserPod, but we decided to prioritize it over Python and Ruby thanks to a concrete use case from a member of our community on
Discord
.
One maintainer of the Yarn package manager expressed interested in building an interactive documentation page, featuring a real yarn build, and running on BrowserPod. But contrarily to previous versions, which were JavaScript, the upcoming
Yarn 6
release is built in Rust.
This is one of the use cases where we think that BrowserPod can really shine, and so we started working on it right away! It took us a week to get to a first prototype and we could immediately see the potential, but there were more moving parts that we had expected.
In previous releases of BrowserPod we focused on Node.js. Our users did not have to worry about compiling for the BrowserPod target, since we provide a pre-compiled Node.js build ourselves.
Rust programs behave very differently, since they need to be compiled ahead of time using
rustc
. As things stand today, the user needs to compile the program offline and add the resulting binary to the Pod. Interestingly,
rustc
itself is written in Rust. In principle, we could allow users to compile their Rust programs directly in the browser. We need a few additional features to achieve this objective, but it will happen in the near future.
Effectively “supporting Rust” today boils down to providing a Rust toolchain that can produce Wasm binaries for the BrowserPod target.
Fortunately, we could start from solid ground. Rust already supports a variety of Wasm targets, namely:
wasm32-unknown-unknown
: This target is the most barebones and platform agnostic. By itself, it makes no assumptions about the environment it runs in, but crates like
wasm-bindgen
and
web-sys
make it possible to communicate with the JS and Web environment. This target makes sense if you are deliberately targeting the Web from the start. Many standard library features are missing.
wasm32-unknown-emscripten
: This target is intended for standalone programs targeting the Web. It can directly interact with JavaScript code, and already wraps many Web APIs for ease of use. It also emulates some POSIX APIs to an extent, but many Rust standard library features are not implemented. In general, supporting this target requires extensive rewriting.
wasm32-wasip1
/
wasm32-wasip2
/
wasm32-wasip3
: These are the
WASI
targets. They can run in principle in any environment, although their main use is for WebAssembly outside of the browser. As such, they don’t assume the presence of JS, but they provide applications with I/O facilities like a filesystem and networking.
wasm32-wali-linux-musl
: This target is intended to run existing Linux applications in a Wasm runtime, by providing the x86-64 Linux syscall interface as imports. Its main use case is outside of the browser, and it is currently mostly an academic project.
Our original plan was to implement a WASI (WebAssembly System Interface) layer on top of BrowserPod. This solution would make any program targeting WASI immediately compatible with BrowserPod, whether it’s written in Rust, Go, C, or anything else.
But when trying to compile existing Rust programs to WASI, we quickly realized that it’s not just a matter of switching build targets.
Take
yarn
for example. Compiling it for WASI would require:
std::os::unix
with
std::os::wasi
: this is doable, although the mapping is not 1:1 (e.g. no absolute symlinks in WASI).
wasm32-wasip3
has some limited cooperative threading support (without real parallelism), but it’s very new and supported by neither the standard library nor
tokio
.
yarn
relies on external programs for some functionality: in particular
git
(to fetch git dependencies) and
node
(to run lifecycle scripts) are required for basic operations.
BrowserPod already solves these problems, so we decided to skip the middleman
and implement our own Rust target:
wasm32-browserpod-linux-musl
.
Defining a
custom Rust target
is
surprisingly easy
: you define your target’s properties
in a .json file:
{
"llvm-target": "wasm32-unknown-unknown",
"target-pointer-width": 32,
"target-c-int-width": 32,
"data-layout": "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-i128:128-n32:64-S128-ni:1:10:20",
"arch": "wasm64",
"is-like-wasm": true,
"target-family": ["unix"],
"os": "linux",
"env": "musl",
"vendor": "browserpod",
"linker-flavor": "wasm-ld",
"linker": "rust-lld",
...
}
And then pass the path of this file as the
--target
argument.
A few unstable features are needed, so we have to use a nightly version of the compiler.
We also need to support our target in the Rust standard library for our combination of arch and os.
Since it’s modeled after x86 Linux, we can mostly copy from that.
And finally, we need to override a couple of foundational libraries:
libc
(we need to link to our own musl libc build) and
linux-raw-sys
(we just select the
libc
fallback).
A number of options to
cargo
and
rustc
are needed to tie everything together (unstable feature flags, the sysroot with the C dependencies, the cargo overrides for
libc
and
linux-raw-sys
, …); we provide wrapper scripts for them, so the user can compile with a simple
cargo build --target wasm32-browserpod-linux-musl
.
In an ideal world, this would be all we need. Unfortunately, we need to deal with the messy reality of third-party dependencies.
As mentioned, Rust already supports multiple Wasm targets.
Many libraries can compile for one or more of those targets, but often with reduced
functionality, or with the assumption that
arch="wasm32"
means “running in the browser’s main thread”,
and other arbitrary constraints.
To reach our goal of compiling projects without changing the code , we need to dodge all the conditional compilation that would mistakenly categorize our target as a “reduced functionality” one.
If you look closer at the json snippet of our target definition above, you will notice two interesting things:
target_family: ["unix"]
: originally, we had
["unix", "wasm"]
, but many crates use the wasm target family to mean “standalone web build”, “WASI”, “no filesystem”, or similar. Dropping “wasm” seems to be
completely harmless
.
arch: "wasm64"
: this is even more surprising, but unfortunately also necessary.
arch="wasm32"
is often used to detect standalone web targets (for example,
reqwest
will replace the whole HTTP client with a
fetch()
implementation). You would think that something would break because of this, but important info like the size of pointers is defined separately, and the actual target passed to the LLVM backend is still
wasm32
, so it all works out!
Of course, we would love to get rid of these hacks, but the proper solution requires awareness in the ecosystem that fully-featured Wasm targets exist, which will take time (but we have a plan ).
The original goal was to run
yarn
, but since our approach was completely general, we found that
a lot of programs
just work
.
Here is a (non-exhaustive) list of popular programs that I tried, but we expect most CLI applications to just work:
| Project | Rust SLOCs | Changes | Status |
|---|---|---|---|
|
burntsushi/ripgrep
|
~40k | no changes | working |
|
yarnpkg/zpm
|
~50k | no changes | working |
|
starship/starship
|
~50k | no changes | working |
|
jj-vcs/jj
|
~200k | no changes | working |
|
openai/codex
|
~1.25M | disabled some sandboxing options | working |
You can play with them in the terminal below:
Codex is fairly large and requires an API key to run, so you can also see it in action in the following video:
Or try it yourself at
https://browsercode.io/agents/codex
.
Rust has a single blessed package manager and build system:
cargo
. This makes it much easier for us to provide a good out-of-the-box developer experience compared to, say, C/C++.
Rust also has an official way to obtain and manage multiple toolchains:
rustup
. We can’t be distributed
through
rustup yet, because we are not an officially supported target, but we can install into its directories, so you can enable our toolchain for a single project in the usual way:
# manually install our toolchain (one day this could be via rustup too)
curl https://rt.browserpod.io/3.0.0/rust/install.sh | bash
# override the toolchain for the current project with rustup
rustup override set browserpod-3.0.0
# build
cargo build --target wasm32-browserpod-linux-musl
# uninstall via rustup
rustup toolchain uninstall browserpod-3.0.0
I explained why the existing Wasm targets were not suitable for us, but I glossed
over
wasm32-wali-linux-musl
.
Despite being aimed at native Wasm runtimes rather than browsers,
WALI (WebAssembly Linux Interface)
has the same core goal as BrowserPod: compile Linux applications
unmodified
to Wasm. The main difference really is that BrowserPod targets the x86 version of the Linux syscalls, while WALI targets the x86-64 version. The choice for BrowserPod comes from the fact that its syscall implementations are shared with
CheerpX
, which is a virtual machine for x86 Linux binaries (you may know it from
WebVM
).
CheerpX is working towards supporting x86-64 binaries, so in the future we could simply adopt WALI as our target (for both Rust and C/C++).
This would have many benefits:
We are very excited about this direction and will say more about it in the future!
Head to
console.browserpod.io
to get an API key. You can find the documentation on
browserpod.io/docs
.
If you have a Rust project you’d like to see running in a browser tab (especially one you expect to break) we’d genuinely like to hear about it. Come find us on
Discord
.
TL;DR: DuckDB v2.0 is coming this fall. In this post, we preview its headline features: DuckDB as a server, triggers, the VARIANT type, asynchronous I/O, a new SQL parser, a new storage format, and much more.
DuckDB v2.0 will be named “Cyanoptera” after the cinnamon teal (Anas cyanoptera), a strikingly reddish-brown duck found in the western Americas.
A major version bump is not something we do lightly, and it is not just ceremony: v2.0 ships a new SQL parser, a new default storage format, a reworked C API, and a small number of carefully chosen breaking changes. But above all, it is a feature release, built from over 10,000 commits since we released v1.5 in March. Where last year was the year of the lakehouse, this release kicks off the year of DuckDB as a server. We previewed many of these features in the “State of the Duck” talk at DuckCon #7 , if you prefer to watch instead of read.
DuckDB is moving rather quickly, and we can only cover a small fraction of the changes here. Condensing all new features down to a shortlist is always a fight over what gets in, and yes, we know that what follows is technically a listicle (Ten Things Coming to DuckDB v2.0, Number Eight Will Shock You). We are not proud of the format, but it works, so here it is, starting with the SQL-level features and working down into the engine.
CONNECT
DuckDB has been an in-process database since day one. But people have asked us – very persistently – for a client/server mode, and we have finally caved. The
quack
extension
implements DuckDB's native protocol for talking to other DuckDBs. It was
released as a preview
shortly before DuckCon #7, graduates to stable in v2.0, and it is a big part of where DuckDB is headed: any DuckDB process can serve its databases over the network, and any other DuckDB can attach to it and route queries there using the new
CONNECT
statement. For example:
CALL quack_serve(
token = 'my_token'
);
quack:
ATTACH 'quack:server.example.com'
AS qk (TOKEN 'my_token');
CONNECT qk;
SELECT count(*) FROM events;
-- executes on the server,
-- results stream back
DISCONNECT;
CONNECT
is the successor to the
remote.query($$...$$)
workaround we showed when Quack was first revealed – we looked at that syntax and said: no, this cannot be it. And
CONNECT
is not limited to Quack: it points your session at any remote database that supports it, and the new remote pushdown optimizer (
#22914
) ships SQL directly to PostgreSQL and MySQL instead of pulling tables over the wire:
CONNECT 'postgres://localhost/mydb';
SELECT count(*) FROM orders; -- runs on the PostgreSQL server
DISCONNECT;
If you have worked with analytical systems in the past, you may assume that DuckDB cannot handle transactional workloads. But DuckDB has been built as a transactional, multi-connection database with full MVCC and transaction isolation since day one. Most users just never needed that in a single-user scenario. It turns out DuckDB handles transactions well: it's fast enough to compete with general-purpose databases like PostgreSQL on quite a few workloads, and the client/server pattern finally lets that machinery shine in multi-tenant, long-running deployments.
Running DuckDB long-term also comes with new challenges, which is why v2.0 pushes on better metrics, logs, and observability (see, e.g., the metrics layer rework in #22799 ) that let you look at a DuckDB instance and see what it is actually doing. People even built standalone clients for the Quack protocol within weeks of the preview. We thought we were extending DuckDB to talk to other DuckDBs; the world said no, no, no, and built their own clients. Who would have thought.
VARIANT
Becomes a First-Class Citizen
The
VARIANT
type shipped in
DuckDB v1.5
, and the way to think about it is JSON on steroids. Basically, imagine if JSON were fast. Like JSON, a
VARIANT
column can store differently-shaped data in every row. Unlike JSON, it is not a text format: DuckDB automatically detects the common structure hidden in your semi-structured data and “shreds” it, so it compresses well in storage and executes fast in queries, all without you ever declaring a schema. This makes
VARIANT
a natural fit for real-time log ingestion, where streams of JSON-ish records share structure but evolve over time.
In v2.0, this pipeline works end to end: shredded execution straight from storage (
#20912
), extraction pushdown into scans (
#22478
), shredded
VARIANT
reading
and
writing for Parquet, and a family of
variant_*
functions:
CREATE TABLE events (payload VARIANT);
INSERT INTO events
VALUES ('{"user": {"id": 42, "tags": ["a", "b"]}}'::JSON::VARIANT);
SELECT variant_type(payload), variant_keys(payload)
FROM events;
SELECT *
FROM events
WHERE variant_contains(payload, {'user': {'id': 42}}::VARIANT);
Longer term, likely soon after v2.0 (but don't hold us to it), we plan to back the regular
JSON
type with
VARIANT
, so existing JSON workloads get all of these benefits without changing a single query.
Triggers have been a long-standing feature request, and DuckDB v2.0 delivers them in full:
BEFORE
and
AFTER
triggers,
FOR EACH ROW
and
FOR EACH STATEMENT
, transition tables via
REFERENCING OLD/NEW TABLE
, multiple triggers per event,
RETURNING
on triggered tables, and
DROP TRIGGER
.
The classic use case is audit tables: something happens in the system, and a trigger records what changed. For example:
CREATE TABLE target (id INTEGER, val INTEGER);
CREATE TABLE audit (id INTEGER, old_val INTEGER, new_val INTEGER);
CREATE TRIGGER trg_audit AFTER UPDATE ON target
REFERENCING OLD TABLE AS o NEW TABLE AS n
FOR EACH STATEMENT
INSERT INTO audit
SELECT n.id, o.val, n.val
FROM o
JOIN n ON o.id = n.id;
INSERT INTO target VALUES (1, 10), (2, 20);
UPDATE target SET val = val * 10 WHERE id <= 2;
SELECT * FROM audit;
| id | old_val | new_val |
|---|---|---|
| 1 | 10 | 100 |
| 2 | 20 | 200 |
Triggers fit naturally with long-running DuckDB services, and we are also planning to use them internally to build several upcoming features. They are fully exposed at the SQL level too, so you can build your own cool stuff with them.
As always, DuckDB's SQL dialect keeps growing. A few favorites from this release cycle:
With
NEAREST
joins
(
#24137
), top-k similarity search becomes a join clause, handy for vector and embedding workloads:
SELECT q.user_id, t.product_id
FROM users q
INNER JOIN products t APPROX NEAREST 2
BY SIMILARITY array_cosine_similarity(q.embedding, t.embedding);
DML inside CTEs
(
#21634
,
#21997
,
#24217
) lets you use
INSERT
,
UPDATE
,
DELETE
, and
COPY
as pipeline steps:
WITH moved AS MATERIALIZED (
DELETE FROM staging RETURNING *
)
INSERT INTO archive SELECT * FROM moved;
Nested schemas ( #23492 , #24222 ) allow schemas within schemas:
CREATE SCHEMA finance;
CREATE SCHEMA finance.reports;
CREATE TABLE finance.reports.q3 (revenue DECIMAL);
The new
variable syntax
(
#21194
) lets you write
$x
anywhere an expression is allowed, no more
getvariable(...)
verbiage
:
SET VARIABLE threshold = 100;
SELECT * FROM orders WHERE amount > $threshold;
The
JSON mutation functions
json_set
,
json_insert
,
json_replace
, and
json_remove
(
#23786
) finally let you modify JSON documents in place:
SELECT json_set('{"a":1}', '$.b', '2');
| json_set('{"a":1}', '$.b', '2') |
|---|
| {"a":1,"b":2} |
And
recursive CTEs with
USING KEY
aggregation
(
#19481
) enable iterative algorithms in pure SQL, backed by the rewritten recursive CTE engine described below:
WITH RECURSIVE tbl(a, b) USING KEY (a, avg(b)) AS (
SELECT 1, 5
UNION
SELECT a, b - 1 FROM tbl WHERE b > 0
)
TABLE tbl;
| a | b |
|---|---|
| 1 | 2.5 |
There is more: SQL-standard
FETCH FIRST 2 ROWS ONLY
(
#23533
),
OVERLAY()
(
#22456
),
UNNEST
in
GROUP BY
(
#23644
), and well-defined
MERGE
/
UPDATE ... FROM
semantics for multi-matched rows (
#24058
).
Interacting with object stores like S3 is central to the DuckDB experience: your data has to come from somewhere, and it often sits in object storage. DuckDB has long been able to read from object stores in parallel, but synchronous access placed a limit on how fast this could go. DuckDB v2.0 introduces asynchronous I/O throughout the engine. We described the design in detail in a dedicated blog post .
Thanks to asynchronous access, the I/O layer now scales independently from the query processing layer, which means far more parallelism for remote reads and dramatically faster queries on network storage. Parquet support came first (
#23662
), with CSV (
#23961
) and DuckDB's own file format (
#24654
) following, along with asynchronous Parquet writes (
#23283
) and new
MMAP
and
DIRECT_IO
modes (
#22988
). Local storage benefits a little too, but network storage is where you will see the big gains.
As with every release, a lot of work went into making your existing queries faster without you doing anything. To pick some highlights: partial aggregates are now pushed below joins ( #22572 ) and redundant aggregations are reused ( #24543 ), the recursive CTE engine has been rewritten ( #22211 ), aggregations now spill to disk when they outgrow memory ( #24499 ), and the Windows CLI got approximately 2.2× faster at multi-threaded result materialization ( #24036 ).
How much faster can this get? Here is a microbenchmark you can run on a laptop: single-source reachability over a graph with one million edges, written as a plain recursive CTE .
CREATE TABLE edges AS
SELECT (range % 100_000)::INTEGER AS src,
((range * 13 + 7) % 100_000)::INTEGER AS dst
FROM range(1_000_000);
WITH RECURSIVE reachable(node) AS (
SELECT 0
UNION
SELECT dst FROM edges, reachable WHERE src = node
)
SELECT count(*) FROM reachable;
| Version | Run time |
|---|---|
| DuckDB v1.5.4 | 4.90 s |
| DuckDB v2.0 (preview) | 0.12 s |
As you can see, DuckDB v2.0 is about 40× faster (!) for the same recursive query.
Row-group pruning has been massively expanded:
min-max indexes (zone maps)
and
Parquet Bloom filters
now skip data for structs, lists, decimals, UUIDs,
IN
filters, and even function predicates:
-- these now prune row groups instead of scanning them:
SELECT * FROM logs WHERE contains(message, 'ERROR');
SELECT * FROM t WHERE substr(code, 1, 3) = 'NL-';
SELECT * FROM 'data/*.parquet' WHERE id IN (1, 5, 9);
Query planning also becomes partition-aware ( #22336 ). Lakehouse formats (DuckLake, Iceberg and plain Hive-partitioned Parquet on S3) are all partitioned, and exploiting that partitioning is often the difference between scanning a dataset and skipping most of it. In v2.0, the planner and optimizer take full advantage of existing partitioning, and partitioned writes have been reworked as well ( #22225 , #22620 ).
DuckDB v2.0 bumps the default storage format version to v2.0.0 ( #22875 ). The headline change is buffer-managed ART indexes ( #21458 , #23605 ): indexes are no longer pinned in memory, which means large indexed tables open instantly and their indexes are paged in on demand.
Column metadata is now loaded lazily (
#22333
), so wide tables open faster too. The
DICT_FSST
string compression method is enabled by default (
#23733
), deletes are stored compactly (
#24336
), and the storage layer performs much stronger corruption validation on read. In short: databases with big indexes and wide tables open faster and use far less memory.
DuckDB has famously always used a parser derived from PostgreSQL's. We have decided that enough is enough: v2.0 ships our own modern, extensible PEG-based parser ( #22194 ), an idea we first explored in our 2024 post on runtime-extensible parsers . This change ties into the extension ecosystem: extensions can now hook into the grammar itself, so expect extensions that expose entirely new SQL syntax. It also brings better error messages with precise source locations, and the first dialect compatibility mode:
SET dialect_compatibility_mode = 'spark';
You should not actually notice anything from the parser swap as we designed it to be compatible with the old one. If you do notice, please file an issue.
Timezone-aware timestamps, calendars, and collations in DuckDB have always been powered by the ICU library. ICU is a fine library, but we only ever used a small slice of it, while still carrying it around in every DuckDB distribution. In v2.0, the ICU library is gone entirely: the
icu
extension now implements timezones, calendars, and collations itself (
#24463
,
#24403
), with the timezone data built directly from the IANA database and compressed down to around 45 kB. Everything keeps working exactly as before:
SELECT '2026-08-14 12:00:00'::TIMESTAMPTZ AT TIME ZONE 'Europe/Paris';
SELECT * FROM names ORDER BY name COLLATE de;
Besides being much smaller and easier to keep up to date, the new implementation is also simply faster. Here's a quick microbenchmark on a MacBook that converts 25 million timestamps to a timezone and filters 5 million strings with a German collation:
| Query | v1.5.4 (ICU) | v2.0 (native) | Speedup |
|---|---|---|---|
ts AT TIME ZONE 'Europe/Paris'
, 25 M rows
|
0.24 s | 0.11 s | 2.2× |
Filter with
COLLATE de
, 5 M rows
|
0.15 s | 0.06 s | 2.6× |
Extensions are one of the best things about DuckDB, but today, most of them, including our own, build against the unstable C++ API . That means extension authors have to re-target and rebuild for every DuckDB release, and community extensions can silently disappear when their authors stop keeping up. DuckDB v2.0 broadens the stable C API far enough that extensions can be written once, built once, published once, and keep working, essentially until the end of time.
To make this sustainable over the long run, the C API is now generated from a declarative, versioned specification (
#24135
): every function in
duckdb.h
,
duckdb_extension.h
, and the extension ABI is described in YAML in the
api_spec/
directory
, with its full lifecycle on record, and CI verifies the committed headers against the spec so API and ABI can no longer drift apart. The release also brings unified symbol versioning (
#24435
), custom allocation handlers (
#23945
), and static linking of C API extensions into your application (
#22251
).
So what does building an extension against the stable C API look like? Here is a complete extension: a single file that registers a vectorized scalar function, compiled once against
duckdb_extension.h
.
#include "duckdb_extension.h"
DUCKDB_EXTENSION_EXTERN
// a scalar function that adds two BIGINTs, one vector at a time
static void AddNumbers(duckdb_function_info info, duckdb_data_chunk input, duckdb_vector output) {
idx_t count = duckdb_data_chunk_get_size(input);
int64_t *a = (int64_t *) duckdb_vector_get_data(duckdb_data_chunk_get_vector(input, 0));
int64_t *b = (int64_t *) duckdb_vector_get_data(duckdb_data_chunk_get_vector(input, 1));
int64_t *result = (int64_t *) duckdb_vector_get_data(output);
for (idx_t row = 0; row < count; row++) {
result[row] = a[row] + b[row];
}
}
DUCKDB_EXTENSION_ENTRYPOINT(duckdb_connection con,
duckdb_extension_info info,
duckdb_extension_access *access) {
duckdb_scalar_function f = duckdb_create_scalar_function();
duckdb_scalar_function_set_name(f, "add_numbers");
duckdb_logical_type bigint = duckdb_create_logical_type(DUCKDB_TYPE_BIGINT);
duckdb_scalar_function_add_parameter(f, bigint);
duckdb_scalar_function_add_parameter(f, bigint);
duckdb_scalar_function_set_return_type(f, bigint);
duckdb_destroy_logical_type(&bigint);
duckdb_scalar_function_set_function(f, AddNumbers);
duckdb_register_scalar_function(con, f);
duckdb_destroy_scalar_function(&f);
return true;
}
LOAD add_numbers;
SELECT add_numbers(40, 2);
For brevity, we skipped
NULLhandling here. See thedemo_capiextension for the full version.
The binary this compiles to keeps working across DuckDB versions. You do not need re-target or rebuild it every time a new DuckDB version comes out. And nowadays, with all the AI tooling around, building an extension has never been easier.
So you have written your extension. But how should you distribute it? Until now, DuckDB could only install extensions from the built-in repositories (
core
,
core_nightly
,
community
, …). In v2.0, you will be able to register your own trusted repositories (
#24777
, currently work-in-progress), so an organization can host and sign its own extensions and have them install and load just like the built-in ones:
SET allow_extension_repositories = 'allowed';
CREATE EXTENSION REPOSITORY my_repo FROM 'https://extensions.example.org';
INSTALL my_ext FROM my_repo;
LOAD my_repo/my_ext;
A repository is a name, a URL prefix, and one or more RSA public keys that are trusted to sign the extensions served from it. The prefix can point at anything DuckDB can read: a local path,
https
,
s3
, you name it. At
CREATE
time, DuckDB fetches the repository's public keys and pins them into the repository definition, printing each key's SHA-256 fingerprint so you can compare it against one published out of band. If you would rather not trust the network at all, you can pass the key directly:
CREATE EXTENSION REPOSITORY my_repo FROM 's3://my-bucket/extensions'
USING PUBLIC KEY '-----BEGIN PUBLIC KEY----- ...';
Pinned repositories survive restarts, support key rotation by trusting multiple keys, and can be audited at any time through the
duckdb_extension_repositories()
table function, or removed again with
DROP EXTENSION REPOSITORY
. Together with the stable C API, the extension story rounds out nicely: write your extension once, sign it, host it wherever you like, and
INSTALL
it anywhere.
These are only a few highlights, and this post is only a preview. Some details may still shift before the release this fall, and there are many more features and improvements that we could not cover here. DuckDB v2.0 will also come with a small set of breaking changes, including the new default storage format and the completed lambda syntax transition, which we will cover in detail in the release announcement.
There have been more than 10,000 commits by many contributors since we released v1.5. We would like to thank our community for the detailed issue reports, feedback, and contributions that shaped this release. If you want a taste before the fall, the preview builds have most of these features today, and if something breaks, you know where the issue tracker is.
Pedro Holanda
The Software Freedom Conservancy has announced that Mark J. Wielaard has been honored with the second annual Distinguished Service Award in Software Freedom for his many years of service to software freedom.
Mark is one of many key FOSS developers who has designed his career so that his employers have funded much of his FOSS work. Nevertheless, Mark continues his volunteer work after hours as a key contributor who maintains Sourceware — the oldest FOSS collaboration and developer infrastructure hosting site in history.
In addition to his work on Sourceware, Wielaard is a member of the DWARF Debugging Standard Committee , the maintainer for Valgrind and elfutils , as well as a contributor to various other GNU projects.
We are experiencing high error rates around 20% for web experiences and api traffic. Archive downloads and raw repository content downloads are experiencing an approximate 50% error rate. Investigations are on-going into the root cause, and updates will continue to be provided as we investigate.
Posted Aug 17 , 2026 - 14:04 UTC
Pull Requests is experiencing degraded performance. We are continuing to investigate.
Posted Aug 17 , 2026 - 13:58 UTC
Issues is experiencing degraded performance. We are continuing to investigate.
Posted Aug 17 , 2026 - 13:46 UTC
We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available
Posted Aug 17 , 2026 - 13:45 UTC
Webhooks is experiencing degraded performance. We are continuing to investigate.
Posted Aug 17 , 2026 - 13:44 UTC
Actions is experiencing degraded performance. We are continuing to investigate.
Posted Aug 17 , 2026 - 13:42 UTC
API Requests is experiencing degraded performance. We are continuing to investigate.
Posted Aug 17 , 2026 - 13:41 UTC
We are investigating reports of impacted performance for some GitHub services.
Posted Aug 17 , 2026 - 13:40 UTC
This incident affects: Webhooks, API Requests, Issues, Pull Requests, and Actions.
|
|
GitHub down again? no PR access | |
| 48 points by yodon 33 minutes ago | hide | past | favorite | 17 comments | ||
|
Githubstatus.com currently says everything is working, but it isn't |
||
new the tree: every take you wrote, still there
1667 asks a model for the next paragraph, then keeps every take it gives you side by side on a tree you can walk back through. Your keys, your endpoint, your files on your disk. No analytics, no telemetry, no account anywhere in the source.
curl -fsSL https://1667.ai/install.sh | sh
Installs 1667 0.9.8. Supports macOS arm64/x64 and glibc 2.17+ Linux arm64/x64. Run 1667 upgrade for later managed releases.
This optional path needs GitHub CLI. It verifies version 0.9.8 before it runs the local file.
d=$(mktemp -d) && (trap 'rm -rf "$d"' EXIT && cd "$d" && gh release download v0.9.8 --repo github.com/1667-ai/1667 --pattern install-stable.sh && gh attestation verify install-stable.sh --hostname github.com --repo 1667-ai/1667 --signer-workflow 1667-ai/1667/.github/workflows/release-npm.yml --source-ref refs/tags/v0.9.8 --source-digest 5548e0565fd424725d1c86493060566e01667c62 --deny-self-hosted-runners && sh install-stable.sh)
$d=Join-Path ([IO.Path]::GetTempPath()) ([Guid]::NewGuid()); New-Item -ItemType Directory -Path $d | Out-Null; try { gh release download v0.9.8 --repo 1667-ai/1667 --pattern install-stable.ps1 --dir $d; if ($LASTEXITCODE -ne 0) { throw 'GitHub release download failed.' }; gh attestation verify (Join-Path $d 'install-stable.ps1') --hostname github.com --repo 1667-ai/1667 --signer-workflow 1667-ai/1667/.github/workflows/release-npm.yml --source-ref refs/tags/v0.9.8 --source-digest 5548e0565fd424725d1c86493060566e01667c62 --deny-self-hosted-runners; if ($LASTEXITCODE -ne 0) { throw 'GitHub attestation verification failed.' }; powershell -ExecutionPolicy Bypass -File (Join-Path $d 'install-stable.ps1'); if ($LASTEXITCODE -ne 0) { throw '1667 install failed.' } } finally { Remove-Item -LiteralPath $d -Recurse -Force }
1667 · demo the lantern keeper
1667 homepage demo The opening write view shows “The lantern keeper,” chapter three, parts 11 through 13. Part 12 is on take 3 of 5: “He did not move toward the stairs. Instead he set the brass compass on the bar between them, and its needle went around twice, slow, like a dog deciding whether to lie down, and stopped pointing at Maren. ‘Has the cliff road ever been walked at night without a light?’ he asked. ‘Not by anyone who came back,’ she said. The needle shivered and held.” Compose: Enter opens the composer. The direction “let the unlit lantern answer” is typed and submitted. The generated next part reads: “The lantern flame bent toward the compass, though no door had opened.” The program then waits. Compare takes: Up moves back through story parts to the five-take passage. Left and Right compare sibling takes without removing any of them. Explore the path map: M opens one row per story part. Circles show sibling takes, the selected take is highlighted, and word counts sit at the right. Arrow keys move through parts and takes before Escape returns to the story. Every key in one place: Question mark opens this complete reference, then Page Down shows the remaining commands: 1667 key reference MOVE — read and navigate ↑ ↓: previous or next row ← →: flip between takes Shift+↑ Shift+↓: nudge the page a line Page Up or Control+U: page up Page Down or Control+D: page down G or Shift+G: first or last part [ ]: previous or next chapter U: undo a chapter break addition or removal WRITE — make the next part Space: continue this part Enter or I: type what happens next R: retake with the same prompt Shift+R: retake and edit the prompt W: write a take yourself E: edit prose and prompt Y or Shift+Y: copy the part or whole line Control+↑ Control+↓: previous prompts in Direct N: start a new story OPEN — panels and views M: map of the whole story F: facts kept for context O: switch story or open the library Colon or Control+P: command palette Comma: generation settings Control+G: wide context details Question mark: this key reference Escape: close what is open Q: quit 1667 SHAPE — arrange what exists D: delete this take and everything below it on the current line T: tag the line here C or Shift+C: chapters or end one here X: actions for this part P: show or hide directions Z: typewriter mode Shift+F: facts rail, automatic or off MAP — while the map is open M: cycle path, tree, and mass A: all takes or sketches Enter: reroute a node or sketch L: follow a tree line or open a mass line S: tree to mass, or cycle mass sorts D or T: prune, tag, or change the path Chapter rows differ; the line below the story explains the active row. Drag selects, Control+C copies, and Escape closes.
how it works
Press Enter to open the composer, type what happens next, and press Enter again to generate the next story part.
What it is
You move through a manuscript, and the model writes a paragraph when you ask it to. Then it stops and waits for you.
Ask for another version of a paragraph and the one you were reading stays where it is. The new take sits beside it, and ← and → flip between them. Every take you generate stays on the tree, and d is the only thing that removes one.
A story is a tree of parts and takes. The story line is the path you have selected through it. Branches you walked away from are still there, and still yours to come back to.
Stories sit in a directory you chose. Export writes Markdown next to them. There is no server to sign in to and nothing to migrate off later.
The tree
Ask for another take and the one you were reading does not go anywhere. It becomes a sibling. Press m and the shape of the whole thing opens up: the line you are on, the forks you made, and the paragraphs you wrote once and walked away from.
Three views, cycled with the same key. Nothing is archived, nothing is pruned on your behalf, and d is the only thing that removes a take.
Path map for “The lantern keeper.” Depth 1 through 13; 23 parts across 4 story lines. One row represents each selected story part. Circles beside rows 3, 5, 8, 11, and 12 show sibling takes. Part 12 is selected at take 3 of 5; part 13 carries the canon-storm tag. The clip moves to adjacent parts, returns to part 12, compares its sibling takes, then returns to take 3. Controls: Up and Down move by depth; Left and Right change take; Escape closes.
No tracking. At all.
Every line below describes code that does not exist. Go and check.
Every request the program makes goes to the endpoint you configured. Nothing comes to us, because there is no server of ours to send it to.
One exception: an optional update check, which ships off . Turned on, it reads a version number and tells you. It downloads nothing and replaces nothing.
1667 generation settings Theme: lantern Compose focus: off Provider: dry-run Base URL: not set Insecure HTTP to a machine you own: off Stored key: not set Profile: Default Model: not set Temperature: 0.7 Maximum tokens: 2,048 Sampling: default Context window: 32,768 Effort: default Cache policy: off; no controls; no time to live Alternate token-probability count: off Default route: Default profile Prose route: same as default Controls: Up and Down move; Left and Right choose; Enter advances; S saves; C checks; Escape closes.
Every destination the program knows about is on this panel, and you put it there. Captured in demo mode, so the connection reads
dry-run
.
Models
1667 includes no model and rents you nothing. Point it at a provider you already pay, or at a model running on your own computer. The request goes straight there.
generation
temperature 0.8
max output tokens 2,048
context window 32,768
cache policy off
defaults, editable behind ,
Your stories stay in plain text files on your computer. Your key goes in a private file that only you can read, never into the story folder.
Keys
These eight cover a whole writing session. There are about 40 in total and ? shows you the rest, but nothing below this line is needed to finish a chapter.
and the program shows you the rest
1667 key reference MOVE — read and navigate ↑ ↓: previous or next row ← →: flip between takes Shift+↑ Shift+↓: nudge the page a line Page Up or Control+U: page up Page Down or Control+D: page down G or Shift+G: first or last part [ ]: previous or next chapter U: undo a chapter break addition or removal WRITE — make the next part Space: continue this part Enter or I: type what happens next R: retake with the same prompt Shift+R: retake and edit the prompt W: write a take yourself E: edit prose and prompt Y or Shift+Y: copy the part or whole line Control+↑ Control+↓: previous prompts in Direct N: start a new story OPEN — panels and views M: map of the whole story F: facts kept for context O: switch story or open the library Colon or Control+P: command palette Comma: generation settings Control+G: wide context details Question mark: this key reference Escape: close what is open Q: quit 1667 SHAPE — arrange what exists D: delete this take and everything below it on the current line T: tag the line here C or Shift+C: chapters or end one here X: actions for this part P: show or hide directions Z: typewriter mode Shift+F: facts rail, automatic or off MAP — while the map is open M: cycle path, tree, and mass A: all takes or sketches Enter: reroute a node or sketch L: follow a tree line or open a mass line S: tree to mass, or cycle mass sorts D or T: prune, tag, or change the path Chapter rows differ; the line below the story explains the active row. Drag selects, Control+C copies, and Escape closes.
What we didn't build
Each of these was considered and turned down.
No analytics, no crash reporter, no install id. There is no such code.
No sync, no server of ours. Stories never leave the folder they are in.
Nothing is generated until you press a key that asks for it.
Type an instruction, get a paragraph. There is no thread to scroll back.
FAQ
No. It writes a paragraph when you ask for one, in the direction you gave it, and then it stops and waits. You choose the take, you cut the line, you decide what the chapter is. The judgement stays yours.
They stay. Every take is written to the tree the moment it exists, and moving to another one changes nothing about it. The map shows you all of them, including the lines you abandoned after a single paragraph. Only d removes a take.
No. There is no analytics code, no telemetry, no crash reporter and no install id in the source. Every request the program makes goes to the endpoint you configured, with one exception: an optional update check, which ships off. Turned on, it reads a version number and tells you. It downloads nothing.
1,667 words a day for thirty days is fifty thousand, which is the pace that finishes a first draft. The number names that pace. Nothing in the program tracks whether you hit it.
In a folder you chose. Run 1667 init in ~/book and that is your project. Export writes the selected story line to Markdown in the same place. Back it up like any other directory. There is no export step to be trapped behind.
Anything behind an OpenAI or Anthropic compatible endpoint. Presets ship for OpenAI, Anthropic, OpenRouter, Ollama, LM Studio, llama.cpp, KoboldCpp, and a custom endpoint. Keys are stored in a private file on your machine and sent only to the provider they belong to. A model running locally needs no key at all.
You need to be able to open one. After that it is arrow keys to move, space to continue, and ? for the rest. There is no shell scripting and no configuration file to hand-edit. Settings are a panel behind the comma key.
Yes. The native Windows x64 release uses the PowerShell Installer. Exit 1667 and run the same command again for a later release.
Nothing. It is Apache-2.0 source with no paid tier, because there is no server to pay for. You pay whatever your model provider charges, directly to them, or nothing at all if you run a model locally.
Open a folder, run it, and write the next paragraph.
curl -fsSL https://1667.ai/install.sh | sh
Installs 1667 0.9.8. Supports macOS arm64/x64 and glibc 2.17+ Linux arm64/x64. Run 1667 upgrade for later managed releases.
This optional path needs GitHub CLI. It verifies version 0.9.8 before it runs the local file.
d=$(mktemp -d) && (trap 'rm -rf "$d"' EXIT && cd "$d" && gh release download v0.9.8 --repo github.com/1667-ai/1667 --pattern install-stable.sh && gh attestation verify install-stable.sh --hostname github.com --repo 1667-ai/1667 --signer-workflow 1667-ai/1667/.github/workflows/release-npm.yml --source-ref refs/tags/v0.9.8 --source-digest 5548e0565fd424725d1c86493060566e01667c62 --deny-self-hosted-runners && sh install-stable.sh)
$d=Join-Path ([IO.Path]::GetTempPath()) ([Guid]::NewGuid()); New-Item -ItemType Directory -Path $d | Out-Null; try { gh release download v0.9.8 --repo 1667-ai/1667 --pattern install-stable.ps1 --dir $d; if ($LASTEXITCODE -ne 0) { throw 'GitHub release download failed.' }; gh attestation verify (Join-Path $d 'install-stable.ps1') --hostname github.com --repo 1667-ai/1667 --signer-workflow 1667-ai/1667/.github/workflows/release-npm.yml --source-ref refs/tags/v0.9.8 --source-digest 5548e0565fd424725d1c86493060566e01667c62 --deny-self-hosted-runners; if ($LASTEXITCODE -ne 0) { throw 'GitHub attestation verification failed.' }; powershell -ExecutionPolicy Bypass -File (Join-Path $d 'install-stable.ps1'); if ($LASTEXITCODE -ne 0) { throw '1667 install failed.' } } finally { Remove-Item -LiteralPath $d -Recurse -Force }
|
|||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Got yourself a dreaded case of the Mondays? Start your week off right by catching up on last week's episode of the Hell Gate Podcast. Listen here or wherever you get your podcasts, or watch our beautiful faces on our YouTube channel :
It's not the oppressive humidity or the rain keeping people indoors on this besodden Monday morning. Are you on the isle of Manhattan? Look around you. Yes, look around you. Are you on its grand avenues, in its steamy subway stations, its parks, once a-teeming? Are you on its Broadways, its Canals, its Dyckmans? Beneath its skyscrapers, above its raging sewers, eye-level with its endless maze of scaffolding?
As you wander the quiet streets, are you wondering where the streaming mass of humanity is? Fear not, for they will return. But now—now it is the time of Empty Manhattan , when the summer becomes too much to bear, so people leave or they just melt into the landscape.
First posts are hard, so I'll start small. Here's two arrows:
⇧ ⇩
If you're on reading this on a relatively recent smartphone, chances are you see two reflected, but otherwise identical arrows. One points up and the other points down. On MacOS (Big Sur 11.6), however, the up arrow is much squatter than the one pointing down. This is a photo of how it renders on my MacBook Pro:
Alex, those are two different arrows. Well yes, in the sense that one is up and the other is down, but they should be otherwise identical. I input two Unicode characters, "Upwards White Arrow" and "Downwards White Arrow", and I expected (reasonably, I think) that the down arrow would have the same proportions as the up arrow, only pointing down. So what's going on here?
Let's start with the basics. Unicode, if you've never never had to think about it before, is the international standard for representing text in computer software. When rendering text, your computer reads a series of Unicode "code points" and then turns those into the glyphs specified by your font. In Latin script, every character you type has a corresponding hexadecimal number—a code point—that gets saved in the computer's memory when you type it.
Because each character in the English alphabet is a single code point, representing English in Unicode is straightfoward.
The uppercase letter "C" is code point
U+0043
, lowercase "r" is
U+0072
, and lowercase "o" is
U+006F
.
Other code points include the semicolon (
U+003B
), the lowercase letter "w" (
U+0077
), and the percent symbol (
U+0025
).
To display the text, the computer will read each code point and display whatever that sequence of code points should represent, in the chosen font.
When I write "Crow", what I'm really writing is:
(
U+0043
)(
U+0072
)(
U+006F
)(
U+0077
)
That's why you can easily change the font on a webpage or a document—your computer has all this text saved as code points, it only has to render them differently. Other languages, where multiple code points might combine to create a single character, are far more complex , but operate under essentially the same principle: Unicode provides the code points, the computer translates that into text using a font.
Back to my messed-up arrows.
The ⇧ is the "Upwards White Arrow" code point (
U+21E7
) and the ⇩ is "Downwards White Arrow" (
U+21E9
).
If you're looking closely at those hexadecimals, you'll see they're two numbers apart.
In between them (
U+21E8
) is, you guessed it: ⇨, the "Rightwards White Arrow."
Well maybe Unicode specifices that these arrows should look different for some reason, and I'm using them wrong. As with all open standards, you can simply go look up the definition. The Unicode consortium has a webpage where you can search the whole standard by code point! Which is how I ended up with a PDF of Unicode characters 2190-21FF (8592-8703 in decimal), a subset of the standard appropriately titled Arrows :
That's a lot of arrows! But wait... compututer, enhance!
Those are the arrows I want! They all look the same! If the Unicode standard suggests they should be the same, why don't my arrows do that? One StackOverflow answer for a different set of arrows posits that the Lucida Grande font (MacOS default) might render the arrows differently. This could have been the case for those arrows, but it would also be a weird thing for a font to do. All these arrows live right next to each other on the standard; there aren't a lot of good reasons to render one of a set differently.
The answer is that the Lucida Grande font does not render the other arrows at all , it only renders the Upwards White Arrow. The other arrows are rendered in an entirely different font, a fallback font called STIXGeneral. You can see this by inspecting the following line in your browser's Dev Tools:
⇧⇩
Depending on your browser (I'm using Firefox) and OS, you might see something like this:
PT Sans is the font used on this website, but it doesn't have either of the arrows, so Firefox looks to my system font, Lucida Grande, and renders the Upwards White Arrow using it. Then, seeing that neither of those two fonts supports the Downwards White Arrow, it switches to a more comprehensive fallback font called STIXGeneral to display the character.
Why render just one of the arrows? According to Unicode CJK & Unihan group chair and Apple Font Developer Dr. Ken Lunde, some fonts implemented just the Upwards White Arrow because it is present in many Traditional Chinese fonts via a different, non-Unicode encoding called Big5 . Lucida Grande presumably supported Big5 encoding, and the Upwards White Arrow glpyh was later mapped to its unicode representation, once the "Arrows" set came out. The creators of that font never actually specifically looked at the "Arrows" set and said "we'll support this, but only the up arrow;" they simply re-used the characters that they had ready to go, to support what they could.
There is a platform-independent solution though, one that renders properly no matter what fonts are installed, as long as they have the Upwards White Arrow. I used this little trick to mimic the Reddit upvote arrow recently. Try inspecting the element below:
⇧ ⇧
Have fun!
Enormous thanks to Twitter users @ken_lunde, @fake_unicode, and @litherum for finding my tweet and tagging various experts to help me explore the issue . If you read this post and would like to be credited by name and bio, let me know.
Update 1: Some commentors have pointed out that it's possible the Upwards White Arrow glyph originally represented the shift key , instead of CJK characters. If you were involved in the creation of the Lucida Grande font and know where it came from, contact me!
Tadalafil – better known by its brand name Cialis – is one of the most commonly prescribed drugs for erectile dysfunction.
But in some biohacking and wellness circles, it's increasingly being repurposed for an entirely different reason.
Online clinics and influential figures are promoting it as a kind of all-purpose longevity drug – delivering benefits for the cardiovascular system, brain and even athletic performance.
The drugs aren't approved for preventing heart attack and stroke, let alone for extending life.
However, the enthusiasm does reflect growing interest among some experts in the field of cardiology and men's health that tadalafil and other drugs in its class could have a legitimate role in preventive health.
"It's a serious area of discussion," says Dr. Robert Kloner , chief science officer at the Huntington Medical Research Institute and a professor at the University of Southern California .
"Cardiovascular disease remains the number one killer – and we have these drugs that may have a potential benefit, but we have to learn a lot more."
Tadalafil belongs to the same class of drugs as sildenafil, i.e., Viagra – with the primary difference being that its effect lasts considerably longer.
While both have been on the market for decades, a handful of large studies, many of them published in recent years, have shown that men taking the drug fare better: They have lower rates of cardiovascular disease and death, and are less likely to develop dementia.
The major studies in this area are observational – and retrospective – meaning they can only show associations, not that tadalafil was the definitive cause.
Want the latest stories on the science of healthy living? Subscribe to NPR's Health newsletter .
But the findings are remarkably "consistent" across the literature, says Kloner, whose lab has studied these medications extensively. "It's a very interesting signal."
The findings have surfaced in the online longevity conversation, and prominent voices – from Stanford neuroscientist and podcaster Andrew Huberman to the immortality-seeking tech mogul Bryan Johnson – have talked up the idea of taking tadalafil for more than sex.
Even some women are getting on board.
"It seemed very harmless," says Kristi Sawicki , who holds a doctorate in molecular oncology and has built a following on social media around longevity.
"If anything it's going to help blood flow and that can be beneficial to our brain and our heart," she told NPR.
Sawicki started taking tadalafil about a year ago.
It was one of the offerings in the online telehealth clinic she used for her GLP-1 prescription. That prompted her to dig into the research herself.
"It's a very well-understood mechanism," she says.
As someone with a family history of heart disease, she saw taking tadalafil as "low-hanging fruit" for cardiovascular health – and has also noticed other benefits, like a better pump in the gym.
While mostly used for erectile dysfunction, both tadalafil and sildenafil are also approved for treating a rare form of high blood pressure affecting the lungs, as well as symptoms of an enlarged prostate in men.
In essence, these medications – known as PDE-5 inhibitors – work by preventing the breakdown of a signaling molecule, which in turn helps relax smooth muscle cells and dilate blood vessels.
This not only improves blow flow in one particular region but also relaxes and widens blood vessels throughout the body, Kloner says.
Sildenafil – approved in the late '90s – was famously developed as a drug to treat chest pain caused by poor blood flow to the heart, until its "side effects" were recognized and drug makers changed course.
There's evidence – mostly from small trials in humans – that PDE-5 inhibitors like tadalafil may benefit blood-vessels. Plus, lab and animal research have suggested these drugs can also have anti-inflammatory effects.
The studies drawing on medical records of men who were prescribed these drugs, primarily for erectile dysfunction, have shown sizable effects.
For example, an analysis by Kloner of more than 8,000 men taking tadalafil found they had a 55% lower rate of dying from cardiovascular causes and a 44% lower rate of overall mortality during the study period.
Another study – this based on more than half a million patients – showed a 32% lower risk of dementia among those taking tadalafil, along with improved mortality.
"We were particularly fascinated by the dementia piece," says Dr. Dietrich Jehle , chair of emergency medicine at UTMB and lead author of the analysis, noting that data from the U.K. has also indicated a lower risk of Alzheimer's disease.
Researchers like Kloner and Jehle emphasize that without prospective studies – ideally randomized trials – these findings need to be read with caution. There could be unknown factors that influence why men who seek out medications for erectile dysfunction are less likely to die and develop certain diseases. Kloner also acknowledges the drugs are not as well studied in women.
The problem, he says, is that pharmaceutical companies aren't incentivized to run these costly studies because generic versions of the drugs are already available.
"It's not something that cardiologists are routinely recommending, I'll tell you that," Kloner says. "But these drugs in general are very safe."
Much of the current marketing around tadalafil comes from direct-to-consumer online clinics, where it's often advertised alongside other trendy wellness and longevity therapies.
For example, AgelessRx – a company that Sawicki promotes in her online content – sells a monthly $70 prescription of low dose tadalafil for longevity purposes.
In an interview with NPR, the company's medical director, Dr. Jenell Decker said she felt the data were strong enough to warrant offering it to both men and women over 40.
"In longevity, that's what you have to do with a lot of things, you have to infer," she says, "because if you wait until a person is already showing symptoms, you're already behind."
But some experts are troubled by this off-label use of the medication, without better data in humans.
"Absence of evidence of harms doesn't mean there isn't harm," says Dr. Steve Nissen , chief academic officer of the Cleveland Clinic Heart, Vascular & Thoracic Institute. "In the absence of randomized controlled trial data, it's pretty hard to justify."
There are signs tadalafil could be gaining wider acceptance outside of the online wellness and optimization world.
The topic came up recently during the latest meeting of an influential medical group, known as the Princeton Consensus Panel, which focuses on men's sexual function and cardiovascular health.
The group of medical experts – which included Kloner and other preventive cardiologists – arrived at consensus language that low dose tadalafil is "appropriate" for a subset of men who are at elevated risk of developing cardiovascular disease due to the build up of plaque in their arteries, according to Dr. Martin Miner , who co-chaired the June conference. (Their formal recommendations are not yet published.)
In an email to NPR, Miner acknowledged some were "hesitant" to endorse this broader use of tadalafil because there were no randomized controlled trials, but ultimately decided to do so, based on the existing observational research.
"This is not for all men," says Miner, who directs the Men's Health Center at Brown University in Providence, R.I., though he does prescribe it for many of his patients who're at higher risk of cardiovascular disease.
"I believe men need all the help we can give them due to the longevity gap in this country," he added.
| Dist. | ID | Release | Package | Date |
|---|---|---|---|---|
| AlmaLinux | ALSA-2026:54541 | 10 | .NET 8.0 | 2026-08-14 |
| AlmaLinux | ALSA-2026:54590 | 10 | .NET 9.0 | 2026-08-14 |
| AlmaLinux | ALSA-2026:54510 | 9 | bind | 2026-08-14 |
| AlmaLinux | ALSA-2026:54576 | 10 | dracut | 2026-08-14 |
| AlmaLinux | ALSA-2026:54571 | 9 | dracut | 2026-08-14 |
| AlmaLinux | ALSA-2026:54486 | 10 | freerdp | 2026-08-14 |
| AlmaLinux | ALSA-2026:54487 | 9 | freerdp | 2026-08-14 |
| AlmaLinux | ALSA-2026:54512 | 10 | gnome-remote-desktop | 2026-08-14 |
| AlmaLinux | ALSA-2026:54343 | 10 | kernel | 2026-08-14 |
| AlmaLinux | ALSA-2026:54650 | 10 | nghttp2 | 2026-08-14 |
| AlmaLinux | ALSA-2026:54662 | 9 | nghttp2 | 2026-08-14 |
| Debian | DLA-4742-1 | LTS | apr-util | 2026-08-16 |
| Debian | DSA-6443-1 | stable | docker.io | 2026-08-16 |
| Debian | DLA-4743-1 | LTS | ironic | 2026-08-16 |
| Debian | DSA-6444-1 | stable | neutron | 2026-08-16 |
| Debian | DLA-4740-1 | LTS | postgresql-15 | 2026-08-14 |
| Debian | DLA-4741-1 | LTS | unzip | 2026-08-16 |
| Debian | DSA-6442-1 | stable | util-linux | 2026-08-14 |
| Fedora | FEDORA-2026-51e9d3c767 | F43 | chromium | 2026-08-15 |
| Fedora | FEDORA-2026-c374680c6a | F44 | chromium | 2026-08-15 |
| Fedora | FEDORA-2026-9f70b173c8 | F44 | jfrog-cli | 2026-08-17 |
| Fedora | FEDORA-2026-d51fe2075c | F43 | jrnl | 2026-08-16 |
| Fedora | FEDORA-2026-1f4ad7617f | F44 | jrnl | 2026-08-16 |
| Fedora | FEDORA-2026-110274a705 | F43 | libgsasl | 2026-08-16 |
| Fedora | FEDORA-2026-f2ced62115 | F44 | libgsasl | 2026-08-16 |
| Fedora | FEDORA-2026-c8cfd2f2f9 | F44 | libsoup3 | 2026-08-15 |
| Fedora | FEDORA-2026-8aaf6f724b | F43 | pdns | 2026-08-15 |
| Fedora | FEDORA-2026-706965c440 | F44 | pdns | 2026-08-15 |
| Fedora | FEDORA-2026-ac51ed6e75 | F43 | pdns-recursor | 2026-08-17 |
| Fedora | FEDORA-2026-707054d631 | F44 | pdns-recursor | 2026-08-17 |
| Fedora | FEDORA-2026-030f3f2029 | F43 | perl-Archive-Tar | 2026-08-15 |
| Fedora | FEDORA-2026-fdc77dd5f5 | F43 | php-pear-PHP-CodeSniffer | 2026-08-15 |
| Fedora | FEDORA-2026-d536e7004b | F44 | php-pear-PHP-CodeSniffer | 2026-08-15 |
| Fedora | FEDORA-2026-7ebec18d52 | F43 | rust-bat | 2026-08-17 |
| Fedora | FEDORA-2026-74c4a6f82e | F44 | rust-bat | 2026-08-16 |
| Fedora | FEDORA-2026-7ebec18d52 | F43 | rust-git-delta | 2026-08-17 |
| Fedora | FEDORA-2026-74c4a6f82e | F44 | rust-git-delta | 2026-08-16 |
| Fedora | FEDORA-2026-7ebec18d52 | F43 | rust-git-interactive-rebase-tool | 2026-08-17 |
| Fedora | FEDORA-2026-74c4a6f82e | F44 | rust-git-interactive-rebase-tool | 2026-08-16 |
| Fedora | FEDORA-2026-7ebec18d52 | F43 | rust-lsd | 2026-08-17 |
| Fedora | FEDORA-2026-74c4a6f82e | F44 | rust-lsd | 2026-08-16 |
| Fedora | FEDORA-2026-7ebec18d52 | F43 | rust-pretty-git-prompt | 2026-08-17 |
| Fedora | FEDORA-2026-74c4a6f82e | F44 | rust-pretty-git-prompt | 2026-08-16 |
| Fedora | FEDORA-2026-7ebec18d52 | F43 | rust-tokei | 2026-08-17 |
| Fedora | FEDORA-2026-74c4a6f82e | F44 | rust-tokei | 2026-08-16 |
| Fedora | FEDORA-2026-de8630b736 | F43 | stunnel | 2026-08-16 |
| Fedora | FEDORA-2026-67c2201ad8 | F44 | stunnel | 2026-08-16 |
| Gentoo | 202608-10 | HTTP-Daemon | 2026-08-14 | |
| Gentoo | 202608-13 | NTFS-3G | 2026-08-16 | |
| Gentoo | 202608-12 | Portage | 2026-08-15 | |
| Gentoo | 202608-15 | PostgreSQL | 2026-08-17 | |
| Gentoo | 202608-14 | X.Org X server, XWayland | 2026-08-17 | |
| Gentoo | 202608-11 | haveged | 2026-08-15 | |
| Gentoo | 202608-16 | nginx | 2026-08-17 | |
| Oracle | ELSA-2026-54542 | OL8 | .NET 10.0 | 2026-08-15 |
| Oracle | ELSA-2026-54538 | OL8 | .NET 8.0 | 2026-08-15 |
| Oracle | ELSA-2026-54550 | OL8 | .NET 9.0 | 2026-08-15 |
| Oracle | ELSA-2026-54654 | OL8 | bind | 2026-08-15 |
| Oracle | ELSA-2026-54210 | OL10 | dhcpcd | 2026-08-15 |
| Oracle | ELSA-2026-54576 | OL10 | dracut | 2026-08-15 |
| Oracle | ELSA-2026-54575 | OL8 | dracut | 2026-08-15 |
| Oracle | ELSA-2026-54571 | OL9 | dracut | 2026-08-15 |
| Oracle | ELSA-2026-54184 | OL9 | grafana | 2026-08-15 |
| Oracle | ELSA-2026-53845 | OL10 | iscsi-initiator-utils | 2026-08-15 |
| Oracle | ELSA-2026-53844 | OL9 | iscsi-initiator-utils | 2026-08-15 |
| Oracle | ELSA-2026-36956 | OL10 | kernel | 2026-08-15 |
| Oracle | ELSA-2026-54343 | OL10 | kernel | 2026-08-15 |
| Oracle | ELSA-2026-54443 | OL9 | kernel | 2026-08-15 |
| Oracle | ELSA-2026-54371 | OL8 | nodejs:24 | 2026-08-15 |
| Oracle | ELSA-2026-47756 | OL9 | openssh | 2026-08-15 |
| Oracle | ELSA-2026-22450 | OL10 | osbuild-composer | 2026-08-15 |
| Oracle | ELSA-2026-54484 | OL9 | python-idna | 2026-08-15 |
| Oracle | ELSA-2026-50778 | OL10 | ruby | 2026-08-15 |
| Oracle | ELSA-2026-50773 | OL10 | ruby4.0 | 2026-08-15 |
| Slackware | SSA:2026-227-01 | proftpd | 2026-08-15 | |
| SUSE | openSUSE-SU-2026:11502-1 | TW | 7zip | 2026-08-14 |
| SUSE | SUSE-SU-2026:23071-1 | SLE-m6.1 | afterburn | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11503-1 | TW | ansible-lint | 2026-08-14 |
| SUSE | SUSE-SU-2026:23127-1 | SLE16.0 | bouncycastle | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11512-1 | TW | cargo-audit | 2026-08-16 |
| SUSE | openSUSE-SU-2026:11513-1 | TW | cargo-c | 2026-08-16 |
| SUSE | openSUSE-SU-2026:11511-1 | TW | chromedriver | 2026-08-16 |
| SUSE | openSUSE-SU-2026:21581-1 | oS16.0 | chromium | 2026-08-16 |
| SUSE | openSUSE-SU-2026:11514-1 | TW | containerized-data-importer | 2026-08-16 |
| SUSE | SUSE-SU-2026:23123-1 | SLE16.0 | dnsdist | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11504-1 | TW | dracut-112 | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11515-1 | TW | ffmpeg-9-libavcodec-devel | 2026-08-16 |
| SUSE | SUSE-SU-2026:23115-1 | SLE16.0 | firefox | 2026-08-14 |
| SUSE | SUSE-SU-2026:23137-1 | SLE16.0 | firefox | 2026-08-14 |
| SUSE | SUSE-SU-2026:23074-1 | SLE-m6.1 | freetype2 | 2026-08-14 |
| SUSE | openSUSE-SU-2026:0287-1 | osB15 | git-cliff | 2026-08-17 |
| SUSE | SUSE-SU-2026:23078-1 | SLE-m6.1 | glib2 | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11516-1 | TW | go1.25 | 2026-08-16 |
| SUSE | openSUSE-SU-2026:11517-1 | TW | go1.26 | 2026-08-16 |
| SUSE | SUSE-SU-2026:23077-1 | SLE-m6.1 | google-guest-agent | 2026-08-14 |
| SUSE | SUSE-SU-2026:23073-1 | SLE-m6.1 | google-osconfig-agent | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11518-1 | TW | gzip | 2026-08-16 |
| SUSE | openSUSE-SU-2026:21573-1 | oS16.0 | himmelblau | 2026-08-14 |
| SUSE | SUSE-SU-2026:3622-1 | SLE12 | java-1_8_0-openjdk | 2026-08-17 |
| SUSE | SUSE-SU-2026:3623-1 | SLE15 | java-1_8_0-openjdk | 2026-08-17 |
| SUSE | SUSE-SU-2026:23066-1 | SLE16.0 SLE-m6.2 | kernel | 2026-08-14 |
| SUSE | SUSE-SU-2026:23068-1 | SLE16.0 SLE-m6.2 | kernel | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11506-1 | TW | kernel-devel | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11519-1 | TW | kubeshark-cli | 2026-08-16 |
| SUSE | openSUSE-SU-2026:11520-1 | TW | kubevirt1.9-continer-disk | 2026-08-16 |
| SUSE | SUSE-SU-2026:23128-1 | SLE16.0 | libXfont2 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23070-1 | SLE-m6.1 | libkrun | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11507-1 | TW | molecule | 2026-08-14 |
| SUSE | SUSE-SU-2026:23076-1 | SLE-m6.1 | net-tools | 2026-08-14 |
| SUSE | SUSE-SU-2026:23118-1 | SLE16.0 | nginx | 2026-08-14 |
| SUSE | SUSE-SU-2026:23140-1 | SLE16.0 | nginx | 2026-08-14 |
| SUSE | SUSE-SU-2026:23130-1 | SLE16.0 | nodejs22 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23131-1 | SLE16.0 | nodejs24 | 2026-08-14 |
| SUSE | openSUSE-SU-2026:21580-1 | oS16.0 | open-iscsi | 2026-08-16 |
| SUSE | SUSE-SU-2026:23075-1 | SLE-m6.1 | perl | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11508-1 | TW | pgadmin4 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23116-1 | SLE16.0 | php-composer2 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23138-1 | SLE16.0 | php-composer2 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23121-1 | SLE16.0 | php8 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23120-1 | SLE16.0 | python-httplib2 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23112-1 | SLE16.0 | python-sh | 2026-08-14 |
| SUSE | SUSE-SU-2026:23134-1 | SLE16.0 | python-sh | 2026-08-14 |
| SUSE | SUSE-SU-2026:23114-1 | SLE16.0 | python-ujson | 2026-08-14 |
| SUSE | SUSE-SU-2026:23136-1 | SLE16.0 | python-ujson | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11509-1 | TW | python3-ansible-compat | 2026-08-14 |
| SUSE | openSUSE-SU-2026:11510-1 | TW | python313-nltk | 2026-08-14 |
| SUSE | SUSE-SU-2026:23119-1 | SLE16.0 | rrdtool | 2026-08-14 |
| SUSE | SUSE-SU-2026:23122-1 | SLE16.0 | rsyslog | 2026-08-14 |
| SUSE | SUSE-SU-2026:23129-1 | SLE16.0 | samba | 2026-08-14 |
| SUSE | SUSE-SU-2026:23109-1 | SLE16.0 | spice-vdagent | 2026-08-14 |
| SUSE | SUSE-SU-2026:23110-1 | SLE16.0 | sssd | 2026-08-14 |
| SUSE | SUSE-SU-2026:23132-1 | SLE16.0 | sssd | 2026-08-14 |
| SUSE | SUSE-SU-2026:23111-1 | SLE16.0 | webkit2gtk3 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23133-1 | SLE16.0 | webkit2gtk3 | 2026-08-14 |
| SUSE | SUSE-SU-2026:23124-1 | SLE16.0 | wireshark | 2026-08-14 |
| SUSE | SUSE-SU-2026:23072-1 | SLE-m6.1 | wpa_supplicant | 2026-08-14 |
Sainsbury’s has paused the use of AI face scanning in one of its stores after a customer was wrongly identified as a shoplifter and ejected from the shop.
“I was embarrassed, mortified even, and felt quite humiliated and powerless,” Matt Arnold, 46, said of his ordeal.
The comedy promoter was buying supplies in the store in East Dulwich, in south-east London , for a standup event at Dulwich Hamlet football club when, after scanning his items and a Nectar card, he was approached by two managers who told him he could not be served owing to an earlier incident. He was then asked to leave and they tried to escort him from the store.
“They simply told me they would be walking me out,” Arnold said. “At that point I did stand my ground and said I would be walking myself out. I was determined to keep some dignity. On this point, if nothing else, they did relent.”
As he left, he saw an overhead CCTV monitor alert with a red circle surrounding his face. He asked the shop staff to keep his shopping in the trolley so his friend could come and pick up the supplies for the comedy night happening soon nextdoor.
“I think they were quite confused by this, understandably, but agreed and my colleague Dave went in to pay for and pick up the shop about five minutes later. There was no pause for thought from the staff, no suggestion that they understood this is not how a shoplifter would behave. Just blindly following the machine’s orders.”
Sainsbury’s head office apologised to Arnold the next day and has paused use of its AI-assisted Facewatch technology in the store while an investigation takes place. But Arnold says the technology should be paused in all stores and he is worried about the extent to which AI is being used in society.
“Anyone could be falsely accused and at some point that will be someone vulnerable, someone with mental health issues like anxiety. It’s inevitable,” he said. “Also, I would worry about the confidence-destroying effect of it happening to a younger person or someone less willing or able to stand up for themselves as I have done.”
The supermarket chain uses facial recognition technology to identify shoplifters and other criminals, and says this is to keep staff safe from abuse. But the tech has been plagued with issues. Last September another shopper, Warren Rajah , was ordered out of a branch in Elephant and Castle, south London, after being wrongly identified by the same Facewatch software. Similar cases have occurred including in Home Bargains and B&M stores.
Arnold said: “Sainsbury’s, and indeed any other company using Facewatch, should disable it immediately, at least until they can guarantee the system works flawlessly. They have announced that they are suspending the surveillance in the East Dulwich store, where my incident occurred, but it’s a centralised system, so why not suspend it in every store? If there is a potential problem in East Dulwich – be it with the AI itself or the implementation of it – there are potential problems in all the stores.”
He said he was concerned about the rise of AI in society. “I don’t like surveillance culture, but I can see how at times it is necessary. What isn’t necessary is allowing machines to make decisions and then expect humans to scuttle off and do their bidding unquestioningly.”
A Sainsbury’s spokesperson said: “We have contacted Mr Arnold to apologise for his experience at our Dulwich superstore. The incident was caused by human error, not the facial recognition technology. Customers can be reassured that the Facewatch system has a 99.98% accuracy rate, and every match is reviewed by a trained manager.”
A Facewatch spokesperson said their technology was not at fault in this case. “A correct alert was sent to the retailer, but was subsequently subject to human error in the way it was handled in store,” they said.
The company said suspending a store was a “precautionary measure” to prevent further alerts while staff underwent additional training.
Sainsbury’s said that alerts remained visible on staff devices for up to an hour and it logged all misidentifications.
We placed a tracking device in a shipment of rare books to see which AI company was buying it, and found an Amazon facility where Amazon scans and destroys books.
Amazon is buying massive quantities of books, scanning them for AI training data, and destroying them in the process.
A 404 Media investigation was able to reveal Amazon’s book buying operation, which hasn’t been previously reported, by placing a tracking device in a rare book we suspected would be acquired by an AI company for training data, and following it around the country to its final destination.
That final destination was an Amazon warehouse in Las Vegas, Nevada. Amazon employees who work at this location say all they do is receive massive shipments of printed books which they then cut the bindings off in order to scan the books more quickly. The printed book is destroyed in the process. The logo of the Amazon team that works at this warehouse, called VGT3, is a dinosaur, brandishing its teeth and with a book in its hands.
“Amazon purchases books through commercial channels to help develop and improve the products and services our customers use,” an Amazon spokesperson told me in a statement.
The world’s AI companies are constantly looking for, and spending extreme resources to locate, more material to train their AI models. With books, that sometimes means destroying them in the process, something that large parts of the public have spoken up against, and which we can now confirm Amazon is doing.
In July, I published a story about booksellers who reported a historical spike in sales starting in the past year. They suspected this spike in sales was due to AI companies acquiring any books they can in search of new training data. Printed books are valuable as training data because a lot of the text they contain is not readily available on the internet, which AI companies have already scraped. The data is also conveniently organized and, if the book was printed before 2022, is guaranteed to be free of AI-generated text, which can make any AI model that is trained on it worse via a recursive process called “model collapse.”
📖
Do you know work at a facility where you scan books? I would love to hear from you. Using a non-work device, you can message me securely on Signal at @emanuel.404. Otherwise, send me an email at emanuel@404media.co.
These booksellers suspected AI companies were behind these large bulk purchases because of the high number of books they were buying, the seemingly random choice of books, and the fact that these buyers, unlike libraries and universities, did not seem price sensitive at all. But booksellers couldn’t say for certain who was behind the large purchases because the marketplaces where they sell their books keep the buyers anonymous. When an order comes in, a bookseller ships the sold books to a warehouse operated by the marketplaces, where books are sorted and then sent to the buyer.
In July, one bookseller told me they received a very large order of around 1,000 books on Biblio, one of these marketplaces. The seller agreed to put an Apple AirTag provided by 404 Media in one of the books included in this order so we could see where the book was going. And by extension, which company, AI or otherwise, was behind this massive order.
404 Media granted the bookseller anonymity because they worried sharing this information would harm their business. Biblio did not respond to a request for comment.
The bookseller sent the shipment to Biblio, and it arrived at a California airport. It then traveled by plane to Milwaukee International Airport in Wisconsin. Later that day, the book traveled to a warehouse belonging to a specialized shipping and distribution company called Trifinity, right outside of Kenosha Regional Airport, about 30 miles south. The book remained there for about two weeks, at which point it began traveling west by what appeared to be a truck. I could see the book travel via the highway and spend a night over at a trucking travel center around Grand Junction, Colorado. The next day, the book arrived at its final destination, an Amazon warehouse called LAS8 in Las Vegas.
LAS8 is one of several large Amazon warehouses in the area, each with a different specialty. LAS7, right across the street, for example, is a fulfilment center, while LAS8 appears to mostly operate as one of Amazon’s “print on demand” operations, which will print and ship books to Amazon shoppers as they are buying them. Initially, I was confused about why the book would arrive there, but Amazon employees who work at this location and who discuss working conditions there with other Amazon employees online explain that the the north end of the LAS8 warehouse, where I saw the book arrived, housed a different Amazon operation with a different code: VGT3.
Many Amazon warehouses have unique symbols to represent that specific site. VGT3’s symbol, painted on the entrance to the site and inside, is of a Tyrannosaurus rex, the massive carnivorous dinosaur, with its mouth open, holding an open book.
“I work at VGT3 here in Vegas, and all we do is scan books,” one Amazon employee wrote on a forum for Amazon workers. “Some are assigned to cut books, and others go to receive where they get books and scan the bar codes. We didn't have rates, but now we do, but it's not stressful.”
“Working at VGT3 is nice all we do is scan books,” another Amazon employee wrote. “It's so cool.”
I saw VGT3 employees online talk about how working in this operation is a good but boring job because workers do the same, easy, repetitive tasks all day. I also saw some discussion indicating that VGT3 jobs are desirable for this reason, and because the warehouse sometimes offered night shifts. Earlier this year, some employees expressed concerns that Amazon would shut down the site because they had worked through the books they had and weren’t getting enough shipments of new books, though the site is still operational today.
That employees are scanning the barcodes or ISBNs on books — a unique serial number given to every published book — before scanning their content gives further credence to another theory put forth by booksellers: AI companies are trying to methodically scan every printed book in the world by working through the list of ISBNs. One bookseller told me they suspected this was the case because the very large orders they were getting never included very rare books that do not have ISBNs.
Elsewhere online in 2024, booksellers who sell their books on Amazon said they received a spike in orders to be shipped directly to the VGT3 location for a customer named “Amazon FC.” This was so unusual given its size they suspected the orders were some kind of scam . An Amazon representative chimed in to say they checked and that the orders were legitimate.
Like other major tech companies, Amazon is developing its own large language models (LLMs). Amazon considers its family of models branded Nova to be “frontier” models, meaning it considers them to be competitive with other cutting edge LLMs from Google, OpenAI, and Anthropic. These models require massive amounts of training data in the form of human-written text. Internally, Amazon also used an AI coding agent called Kiro to develop its own software.
We first learned that AI companies wanted to scan millions of books for training data because of a lawsuit from book authors against Anthropic, which revealed Anthropic’s “Project Panama.” The goal of the project was to acquire books from commercial bookselling marketplaces, cut the spines off the books, and scan them. It’s possible to scan books without destroying them, but cutting the spine makes it cheaper and faster. Additionally, the judge in the lawsuit ruled it was fair use and not a copyright violation for Anthropic to scan a book for training data in part because it destroyed the original, printed copy. Essentially, it’s the customer’s right to take physical media and store it digitally, and destroying the original copy means that copy isn’t duplicated and resold, and isn’t cutting into the publisher’s business.
We’re not revealing the titles of the books included in the shipment we tracked, but they are rare, meaning there are not many copies of them in circulation. Sometimes that’s because not many copies of them were ever printed, and sometimes because they are in a foreign language not many people speak. As the bookseller who sold them told me, there are not many people in the world who would care about them in the same way people might care about the first edition of Oliver Twist , but that doesn’t mean they’re not valuable.
“There are different types of value,” the bookseller said. “There's monetary value, obviously, but there are a lot of other types of value. There's historical value, intellectual value, sentimental value. All sorts of things, and all of those the AI companies don't care about. They just want the content as a bunch of words strung together.”
Amazon provided its statement above but did not respond to questions about why it cut the books it scanned, how many facilities like this it has across the world, and their response to the backlash from people upset that AI companies are destroying books.
Those who have been reading this blog or following me on X know that I tend to jump between side-projects. A while back, I made a conscious decision to allow myself to follow my motivation and explore new ideas, because I think it's important for side-projects to feel fun, and never become a chore. That being said, every once in a while I find myself thinking about a project that I set aside a while back, and how I could push it further.
Last year, I wrote a series of blog posts about Plush , which is a toy Lox-like language I created. I put it together to play with different interpreter and virtual machine design ideas. Notably, it has actor-based parallelism, and it's designed so that there is no global VM lock on any critical path, and no situation in which the entire VM has to pause for anything. Later on, I implemented some basic optimizations in the Plush interpreter, and then I wrote a copying Garbage Collector (GC) for the VM. The GC itself is nothing special, but what makes it kind of cool is that each actor has its own fully independent GC. Each actor can run a collection cycle without any synchronization being involved whatsoever. What's a bit unfortunate, though, is that the performance of this GC ended up being pretty disappointing.
I had a personal goal for the Plush GC. I wanted it to be able to collect one million live objects in under 20 milliseconds, with the idea that this would be fast enough to build a 3D game engine in Plush without GC pauses ever being noticeable. I wrote a small
gc_many_objs.psh
microbenchmark that allocates a linked list with a million nodes and then triggers GC in a loop, but the performance came nowhere near my goal. On my MacBook Air M5, the collection time of this implementation comes to roughly 117ms, which is several times too slow. The reason is that I took a convenient shortcut in implementing my copying GC. A traditional
Cheney copying collector
copies objects from one memory block (the from-space) to another (the to-space), and it uses a forwarding pointer that lives in the header of each object, while also using the to-space as a work list to transitively traverse the graph of live objects during the copying process.
In Plush, each actor has its own private allocator that it uses to allocate objects, as well as a message allocator that's used as a buffer to receive messages from other actors. When an object is sent as a message, the sender copies it into the receiver's message allocator. This exists to decouple the sender from the receiver. It means the sender and receiver don't have to lock and synchronize for messages to be exchanged. I wanted to be able to reuse one copying algorithm for both the GC and for copying messages into the receiver's message allocator. For that, I didn't want to use forwarding pointers from the sender's heap, which would mutate objects in the sender. Instead, I used a hash map which tracks the correspondence between objects and their copies. I thought this wouldn't have too much of a performance impact, because hashing pointers is fast, but I was wrong.
My friend and colleague Laurent Huberdeau pointed out something basic that I didn't know until that point, which is that the default Rust
HashMap
uses a secure hashing function, designed specifically to protect against HashDoS. This doesn't affect its functionality, but it does affect performance. Thankfully there's an equivalent
FxHashMap
in the
rustc_hash
crate, which is maintained by the rust-lang project and is a drop-in replacement. Laurent also found a redundant hash table lookup which could be avoided. These simple changes made the copying GC run more than twice as fast, down to 43ms on my M5 laptop. Much faster, though still far from my original goal of 20ms.
Profiling shows that most of the overhead still comes from the hash table. There is worse news, though: the forwarding pointer hash table itself takes up more space than the live data being copied during collection. It makes sense if you think about it. We're copying a linked list. The list nodes are pretty small, with only a next pointer and a value field in each. The hash table entries themselves are a pair of pointers, but what's more, a hash map needs some amount of extra capacity (empty slots) to perform well, otherwise you can run into hash collisions and performance collapses. On top of that, hash functions are meant to be unpredictable. The output should appear to have a quasi-random distribution. If you think about it, that's actually terrible from a cache performance perspective. It means that during the GC, we end up touching memory all over the place, more than the data we're copying, in an unpredictable pattern. Not great.
There are other inefficiencies in this GC. In a traditional Cheney GC, the to-space is traversed linearly and serves as a work list. We use the to-space itself to keep track of which objects we've copied and then we traverse the pointers in these objects to copy other objects that are also live. If you don't have that, then you need to keep a separate work list. This can be a simple dynamic array that serves as a stack. It's not the end of the world, but it can also add extra allocations, extra memory usage and memory accesses, etc. The worst part of my implementation, though, is that after objects were forwarded, I traversed the hash map a second time to go through the forwarded objects and replace pointers to from-space objects with pointers to their copies in the to-space. However, as stated earlier, the hash map stores pointers in a quasi-random order, so now we're accessing the from-space and the to-space in an unpredictable order as well. Welp.
I think that somewhere in my mind I kind of got used to the assumption that hash maps are an efficient data structure. Introductory CS classes will teach you that you can get
O(1)
time complexity on average. They work well for so many uses. If you're trying to optimize memory usage and cache-friendliness for maximum throughput, though, it turns out that maybe they're not. I originally said that the reason I didn't want to use forwarding pointers is that I was using the same copying algorithm to copy objects when sending messages, and I didn't want to overwrite objects (or object headers) from the sender's heap during that process. There's a simple solution for that problem though, which is that for this special case, we can keep a list of forwarded objects, and come back to undo the forwarding pointer writes after copying. Sounds inefficient, but in practice, messages sent to other actors are probably not massive graphs of objects most of the time, and normal GC use can just skip this step.
At this point I decided to rewrite the Plush GC to simply follow the traditional Cheney copying algorithm, with a toggle that allows us to store an undo-list to remove forwarding pointers and restore object headers for the message send special case. This brings our GC time for a million live objects all the way down to 7ms, which is about 16.7x as fast as the naive implementation we started with. That's an amazing performance improvement, and it's well below my 20ms goal. In fact, I have an example program that renders a rotating cityscape with about 2200 polygons. It triggers GC regularly because it does 3D vector and matrix operations and allocates tons of temporary objects. For this program in particular, the GC time is below 1ms.
Just for historical context, Cheney published
a paper
about what is now known as the Cheney algorithm in 1970. At the time, he was working on a
Ferranti Atlas 2
computer. This was a transistorized supercomputer from the early 1960s. It occupied an entire large room, used core memory, and surprisingly, already had an early form of cache. Regardless of cache efficiency though, memory was a precious resource back then, and using a forwarding pointer is much more memory-efficient than using an auxiliary data structure. I hope that this conclusion isn't too underwhelming, because we've sort of gone full circle to the conclusion that the original Cheney GC algorithm with forwarding pointers is much more efficient. Another indication that we should respect the wisdom of our elders and their sacred publications. Still, I think it's good to understand what, exactly, makes something efficient or not, and how much of a difference things like cache efficiency and predictable memory access patterns can make. It's also good to know that Rust's
HashMap
traded a security footgun for somewhat of a performance footgun.
In addition to making the GC run faster, I made another improvement, which serves both to remove a restriction in Plush and to reduce memory usage. Previously, I didn't have any logic to grow an actor's message allocator. This meant that message size was limited to 16MB, a hardcoded constant. The message allocator uses bump pointer allocation like a normal GC heap, and it's slightly tricky to resize it because if you reallocate the backing storage, that invalidates pointers to queued messages. This means you can only reallocate the backing storage when the queue is empty, and all messages have been consumed by the receiver. That, in turn, requires senders and the receiver to coordinate. If one actor is trying to send a large message, it would need to communicate to the receiver that its message allocator needs to be upsized, and meanwhile, all other senders would have to wait. There's a simpler solution though.
There's a cool
mmap
trick that I believe I learned from Alan Wu. This has been used in YJIT, my own UVM project, and also in many other runtimes. You can use
mmap
to pre-reserve a large contiguous chunk of virtual address space with the
MAP_PRIVATE | MAP_ANONYMOUS
flags and
PROT_NONE
protection. This is essentially telling the OS not to map any other memory or resources in this chunk of virtual address space, but the memory is not physically backed by any RAM, and so it uses up no RSS. Later on, you can come back and
mprotect
pages from this space as
PROT_READ | PROT_WRITE
to make them accessible to your program. This gives you zeroed memory that you can read and write, but the OS doesn't map these pages to physical RAM until you write some data there.
The thing to know is that virtual address space is very large, currently 128TB (that's terabytes) on macOS and Linux (based on 48-bit addressing), and so the size of your initial reservation can be very generous. Even 512GB is only a tiny fraction of the 128TB available. The implication here is that you can essentially have the equivalent of a C++
std::vector
or a Rust
Vec
that can be dynamically resized at will, but you can always take pointers inside this dynamically-sized vector. Resizing the vector and increasing its capacity doesn't invalidate old pointers. You can even shrink that vector, returning pages to the OS, without changing any addresses. Cool, huh? In the context of Plush, this means that a sender can trivially grow the receiver's message allocator, without any coordination with the receiver or other senders being involved. The receiver can later shrink its message allocator if it has grown too big, returning memory to the OS, without needing to consume all messages in the queue.
With the rewritten GC and the
mmap
trick, Plush has a GC that is not only much faster (presumably good enough for a real-time game with lots of allocations), but also uses less memory. The collection itself doesn't use a bulky hash map, but the baseline RSS is also much smaller. A trivial program with no actors uses 9.6MB of peak RSS and starts up in less than 10ms. A trivial program with 2000 actors uses 224MB of peak RSS and starts up in 0.23s. Not bad.
In terms of next steps, I have several ideas on how to improve the performance of Plush, or bring it a bit closer to a "real" programming language. One thing that stands out is that Plush uses a Rust tagged union to represent its
Value
type (also known as a "fat value" representation). This makes for nice readable code, but it also means we're using 16 bytes per value when alignment is taken into account. Most dynamic languages use a tagging scheme. That comes with some compromises, but it could shrink memory usage quite a bit. Plush also uses a stack-based interpreter, whereas a register-based interpreter could be much faster. I've also been wondering how feasible it would be to get an LLM to write a naive JIT compiler for Plush.
As a side note, you may have been wondering why I chose a copying GC for Plush. Maybe a mark-and-sweep GC would actually be faster. My motivation was that copying GCs can do very fast bump allocation (great for a dynamic language that allocates a lot), and they have the neat property that collection time is proportional to the live data being copied rather than total heap size. There's also a theoretical cache advantage with related data being close together in memory. It could be that those assumptions are wrong and mark-and-sweep can win. If you're curious and want to play around I added
benchmarks/gc_many_objs.psh
and
benchmarks/gc_alloc_speed.psh
to
the Plush repo
. Your favorite coding agent can potentially refactor Plush to use a mark-and-sweep GC in less than 20 minutes. If someone wants to try that experiment, I would be curious to know the result. Just make sure to run the benchmarks with
cargo run --release
and also run
cargo test
to make sure that the tests still pass.
Subscribe to my mailing list and follow this blog:
Copyright © 2011–2026 Maxime Chevalier-Boisvert. All rights reserved.
Sokoban ("warehouse keeper") is a 1980s puzzle: push every box onto a goal. In this variant the keeper must also finish on a goal.
Moves: 0 Optimal: –
Keeper (you) Box Goal Box on goal Wall
The warehouse is a grid. On each step the keeper moves one square up, down, left or right. The keeper cannot walk into a wall or a box. It can push a single box if the square just beyond the box (in the push direction) is empty floor or a goal. Only one box moves per step, and a box can be pushed out of a goal again to make room.
W A S D
, or the on-screen pad.
Undo
steps back.
Reset
restores the board.
Sokoban is an A* search problem, but a naive version that explores one keeper step at a time explodes on crowded boards. What runs here is a plain-JavaScript port of a native C++ optimal solver I wrote. It returns the provably fewest-moves solution, not just some solution:
Boards 1–14 are solved live to the proven optimum in milliseconds (the move counts shown as "Optimal" above are exactly what this solver returns). Board 15. The 8-box maze. Is the exception: its optimal search explores ~49 million states and needs >1 GB , which would take far too long to run inside a browser tab. So its optimum ( 184 moves ) was computed offline by the native C++ build of this exact algorithm (a parallel A* search, ~5 s across 24 cores) and verified by replay, and the page simply plays that precomputed solution back . That is why board 15's answer is hardcoded rather than searched here.
Built from my Sokoban solver . About Sokoban →
I work with industrial infrastructure that requires nanosecond-accurate time synchronization. The commercial GPS-disciplined PTP grandmaster clocks that solve this problem run from several thousand dollars on the low end, and considerably more with options and support. I had CM4s and CM4 IO boards already sitting in the lab. A TimeHAT with an OCP M.2 GNSS module would have been the cleaner path at around $400, but that is steep when the goal is learning how PTP actually works, not deploying production infrastructure. This build came in at around $103 in new parts.
This post documents what I built, what broke, what I had to fix, and the results I measured.
Time precision is a spectrum:
| Unit | What it is | Example Uses |
|---|---|---|
| 1 second (s) | The tick of a clock | Human scheduling, cron jobs |
| 1 millisecond (ms) | 1 second split into 1,000 pieces | Web browsing, video streaming, log timestamps |
| 1 microsecond (µs) | 1 second split into 1,000,000 pieces | High-frequency trading, audio/video sync, distributed databases |
| 1 nanosecond (ns) | 1 second split into 1,000,000,000 pieces | Aerospace instrumentation, industrial automation, scientific data acquisition, this build |
NTP is good enough for web servers, databases, authentication systems, and anything where you need to know when something happened but don’t need to coordinate hardware events across machines. Over a LAN with a good reference, NTP can reach low microseconds. Over the WAN it typically lands in the low milliseconds. The ceiling is the network path jitter between client and server, every variable delay adds noise to the offset estimate.
PTP (Precision Time Protocol, IEEE 1588) handles the nanosecond tier. With hardware timestamping at the Ethernet PHY layer, PTP can synchronize clocks to within tens of nanoseconds on a local network. The grandmaster clock is the root time source. It takes GPS-disciplined time and distributes it to clients via the PTP protocol.
A nanosecond is the time it takes light to travel about 30 centimeters (roughly 1 foot). After this build, the clocks on my homelab nodes were off from GPS truth by the time it takes light to travel across a room.
| Component | Notes | Cost |
|---|---|---|
| Raspberry Pi CM4 (8GB/32GB eMMC) | Already owned | N/A |
| CM4 IO Board | Already owned | N/A |
| SR1723U10 GPS module | u-blox M10 based | ~$15 |
| JEFA Tech U.FL to SMA pigtail | For antenna connection | ~$8 |
| Bingfu GPS antenna | Active, SMA | ~$12 |
| MOOKEERF SMA cables | Various lengths | ~$10 |
| Waveshare CM4 IO Board case | 1U aluminum | ~$25 |
| ELEGOO Dupont jumper wire kit | For wiring | ~$8 |
| M2.5 standoff kit | Mounting hardware | ~$8 |
| 12V 2A power supply | For IO board | ~$12 |
| CR2032 battery | For onboard RTC | ~$5 |
| Total (new parts) | ~$103 |
The CM4 and IO board were already on hand.
This build vs the TimeHAT path:
| This Build | TimeHAT Path | |
|---|---|---|
| Compute | CM4 + IO Board (owned) | Raspberry Pi 5 (~$80) |
| GPS integration | SR1723U10 + wiring (~$23) | TimeHAT + OCP M.2 GNSS module (~$395) |
| Antenna | Bingfu + pigtail (~$20) | Same antenna works (~$20) |
| Oscillator | Standard crystal | TCXO (temperature compensated) |
| New spend | ~$103 | ~$475+ |
The oscillator difference matters. The TimeHAT includes a TCXO, which compensates for temperature-induced frequency drift. During GPS holdover, a TCXO holds time significantly more accurately than a standard crystal because its frequency stays stable as the board heats up or cools down. The CM4 uses a standard crystal with no temperature compensation. This build held 15ms of drift over 10 hours of holdover, which was acceptable for this use case. In a production environment where holdover accuracy is critical, the TCXO is worth the cost difference.
The TimeHAT also uses an Intel i226 NIC rather than relying on the CM4’s onboard BCM54210PE, uses proper SMA connectors, and requires no jumper wires. If you are starting from zero hardware, that path is easier. If you have a CM4 already and want to understand what is happening at the PHY level, this build gets you there for a fraction of the cost.
The key hardware fact: The CM4 uses the BCM54210PE Ethernet PHY which has full IEEE 1588v2 hardware timestamping support. This is what makes sub-microsecond PTP possible on the CM4. The BCM54210PE timestamps packets right at the wire inside the PHY, before the packet touches the kernel network stack. This eliminates the jitter that comes from kernel scheduling and interrupt handling.
Clients:
The Turing Pi 2 has two external RJ45 ports. Both ports are bridged into the same switch fabric as the four node slots. The grandmaster is plugged into ge1 (the second RJ45 port). The RK1 and CM4 client nodes are on the internal node slots. All five are on the same RTL8370MB-CG+ switch fabric, which means PTP packets between the grandmaster and clients pass through a single switch hop with no routing involved.
The CM4 was originally running Ubuntu 22.04 with kernel 5.15.0-1093-raspi. The BCM54210PE PTP support was added to the Raspberry Pi kernel tree in 2022 and subsequently upstreamed to mainline Linux, but Ubuntu 22.04’s raspi kernel did not include it. The result was no PHC device and no hardware timestamping:
$ ethtool -T eth0
Time stamping parameters for eth0:
Capabilities:
software-transmit
software-receive
software-system-clock
PTP Hardware Clock: none
$ ls /sys/class/ptp/
(empty)
Without
/dev/ptp0
, SatPulse has no hardware clock to discipline. There is no path from GPS to network time.
There was also a secondary issue: Ubuntu 22.04 had
console=ttyAMA0,115200
in
cmdline.txt
, attaching the kernel console to the same UART the GPS module uses. This caused continuous input overruns in dmesg and garbage output from the GPS port.
Upgrading to Ubuntu 24.04 with kernel 6.8.0-raspi fixed both problems:
$ ethtool -T eth0
Time stamping parameters for eth0:
Capabilities:
hardware-transmit
hardware-receive
hardware-raw-clock
PTP Hardware Clock: 0
$ ls /sys/class/ptp/
ptp0
The SR1723U10 connects to the CM4 IO Board via five Dupont wires. Four carry power and serial UART. The fifth carries the PPS signal to the J2 header.
| SR1723U10 pin | Color | CM4 IO Board |
|---|---|---|
| VCC | Red | 40-pin pin 1 (3.3V) |
| GND | Brown | 40-pin pin 6 |
| TX | Orange | 40-pin pin 10 (RXD0) |
| RX | Yellow | 40-pin pin 8 (TXD0) |
| PPS | Green | J2 header pin 9 (SYNC_OUT) |
The J2 header is the small 10-pin header on the CM4 IO Board near the Ethernet jack. Pin 9 is labeled SYNC_OUT and connects to the BCM54210PE PHY’s external timestamp input. This is how the GPS pulse-per-second signal gets into the hardware clock.
Add to
/boot/firmware/config.txt
:
dtoverlay=disable-bt
enable_uart=1
dtoverlay=i2c-rtc,pcf85063a,i2c_csi_dsi
The first two lines disable Bluetooth and connect
/dev/ttyAMA0
to the GPIO header so the GPS module can communicate with the CM4. The third line enables the onboard PCF85063A real-time clock on the correct I2C bus for the CM4 IO Board.
Remove the serial console from
/boot/firmware/cmdline.txt
:
# Remove this portion from cmdline.txt:
console=ttyAMA0,115200
Disable the Bluetooth service. The
dtoverlay=disable-bt
overlay disconnects Bluetooth from the UART hardware, but the
hciuart
service will still try to initialize and may interfere:
sudo systemctl disable hciuart
After reboot verify the GPS module is outputting NMEA at 38400 baud:
(stty 38400 -echo -icrnl; cat) </dev/ttyAMA0 | head -10
You should see lines starting with
$GNRMC
,
$GNGGA
, and similar. The SR1723U10 defaults to 38400 baud, not the 9600 that many guides assume.
The stack is SatPulse + ptp4l + chrony. SatPulse is a daemon written by jclark that ties the GPS module, the PHC, ptp4l, and chrony together. It handles GPS configuration via UBX protocol, reads PPS timestamps from the PHC, disciplines the PHC frequency, and feeds samples to chrony via a SOCK refclock.
Install SatPulse from the GitHub releases page (arm64 .deb):
wget https://github.com/jclark/satpulse/releases/download/v0.3.0/satpulse_0.3.0_arm64.deb
sudo dpkg -i satpulse_0.3.0_arm64.deb
Configure SatPulse at
/etc/satpulse.toml
:
[phc]
interface = "eth0"
[serial]
speed = 38400
[gps]
config = true
[ntp]
sock.path = "/var/run/chrony.satpulse.sock"
[ptp]
ptp4l.udsAddress = "/var/run/ptp4l"
Install linuxptp and configure ptp4l as grandmaster at
/etc/linuxptp/ptp4l.conf
:
[global]
masterOnly 1
tx_timestamp_timeout 100
ptp_minor_version 0
[eth0]
Copy the SatPulse ptp4l service file:
sudo cp /usr/share/doc/satpulse/ptp4l.service /etc/systemd/system/ptp4l.service
sudo systemctl daemon-reload
sudo systemctl enable --now ptp4l
Configure chrony at
/etc/chrony/conf.d/satpulse.conf
:
refclock SOCK /var/run/chrony.satpulse.sock poll 2 filter 4 refid GNSS
Add an allow directive to
/etc/chrony/chrony.conf
so clients can use the grandmaster as an NTP server:
allow 192.168.0.0/24
log tracking measurements statistics
Restart chrony:
sudo systemctl restart chrony
Enable and start SatPulse:
sudo systemctl enable --now satpulse@ttyAMA0
This is where it got interesting.
Every time the system boots, the BCM54210PE driver initializes the SYNC_OUT pin as output (
1 0
) instead of input (
0 1
). SatPulse tries to reconfigure it but fails silently. The result is
no PTP hardware clock external timestamps being received
and a completely non-functional grandmaster.
Diagnose it:
cat /sys/class/ptp/ptp0/pins/SYNC_OUT
# "1 0" means the pin is stuck in output mode
sudo satpulsetool sdp -i --pin 0 eth0
# "no timestamps received" means the PHC is not getting PPS
Fix it by forcing the pin into the correct state before SatPulse starts. Create
/etc/systemd/system/satpulse@.service.d/override.conf
:
[Unit]
After=chrony.service ptp4l.service
Requires=chrony.service
[Service]
ExecStartPre=/bin/sh -c 'sleep 5 && echo 1 0 > /sys/class/ptp/ptp0/pins/SYNC_OUT && sleep 1 && echo 0 1 > /sys/class/ptp/ptp0/pins/SYNC_OUT'
The toggle from output to input forces the driver to properly reset the pin state. The
sleep 5
gives chrony and ptp4l time to finish starting before SatPulse connects to their sockets.
The CM4 IO Board has a PCF85063A RTC with a battery holder. Without
dtoverlay=i2c-rtc,pcf85063a,i2c_csi_dsi
in
config.txt
the kernel does not know the RTC exists. On every reboot the clock reset to the last known time from before the reboot, which in one case was weeks behind the current time. Chrony then stepped the clock by that amount. SatPulse failed because the PPS timestamps it was receiving were weeks ahead of the system clock.
After adding the overlay, set the RTC to system time:
After the next reboot, verify it holds:
date
sudo hwclock --show
# Both should show approximately the same current time
Without explicit ordering, SatPulse can start before chrony or ptp4l creates their Unix sockets. The
After
and
Requires
directives in the override file above handle this.
Install linuxptp on each client. Ubuntu 24.04 ships the package but does not include a systemd service file. Create
/etc/systemd/system/ptp4l.service
:
[Unit]
Description=Precision Time Protocol (PTP) service
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/sbin/ptp4l -f /etc/linuxptp/ptp4l.conf
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
CM4 client (linuxptp 4, Ubuntu 24.04)
at
/etc/linuxptp/ptp4l.conf
:
[global]
clientOnly 1
tx_timestamp_timeout 200
ptp_minor_version 0
[eth0]
RK1 client (linuxptp 3, Ubuntu 22.04)
at
/etc/linuxptp/ptp4l.conf
:
[global]
slaveOnly 1
tx_timestamp_timeout 100
[eth0]
Configure chrony on both clients at
/etc/chrony/conf.d/ptp.conf
:
refclock PHC /dev/ptp0 poll 0 dpoll -2 tai refid PTP
The
tai
keyword tells chrony the PHC is in TAI timescale and applies the 37-second UTC offset automatically.
Add the grandmaster as an NTP source for comparison:
server 192.168.0.6 iburst minpoll 0 maxpoll 4
Update makestep so chrony steps the clock immediately after large offsets instead of slewing slowly:
makestep 1 -1
When everything is working, SatPulse summaries look like this:
satpulsed: summary: absOffMax=18 absOffMean=6.4 offRMS=7.8 freqMean=3533 nMissing=0 nOutliers=0
The grandmaster PHC is within 18 nanoseconds of GPS truth, mean offset 6 nanoseconds.
With the grandmaster serving both PTP and NTP, the clients see all three tiers simultaneously.
| Source | Offset | Uncertainty |
|---|---|---|
| PTP, hardware timestamping | 3–4 ns | ±66–291 ns |
| LAN NTP, from this grandmaster | +31 to +108 µs | ±79–172 µs |
| WAN NTP, public pools | +1 to +3 ms | ±40–82 ms |
The grandmaster is the same physical machine serving all three. The difference is entirely in how time is delivered and measured. The +/- values are the uncertainty bounds reported by chrony: the range within which the true offset likely falls. PTP uses hardware timestamps in the PHY at the wire. LAN NTP uses software timestamps in the kernel, so it picks up jitter from kernel scheduling and interrupt handling. WAN NTP adds network path jitter on top of that.
Each tier is roughly 1000x worse than the one above it.
The CM4 client (k8s1) consistently achieves +/-66ns uncertainty. The RK1 achieves +/-291ns. Both are good results, but there is a 4x gap.
The CM4 client uses the same BCM54210PE PHY as the grandmaster. It timestamps at the PHY layer, right at the wire. The RK1 uses a Rockchip GMAC with a different PHC implementation. The timestamping happens at the MAC layer rather than the PHY layer, one step further from the wire. Every additional stage between the packet arriving at the silicon and the timestamp being captured adds jitter. The +/-291ns floor on the RK1 reflects that.
#* PTP 0 0 377 1 -3ns[ -4ns] +/- 291ns <- RK1
#* PTP 0 0 252 2 +3ns[ +3ns] +/- 66ns <- k8s1 (CM4)
The grandmaster plugged into a TP-Link AXE5400 consumer router vs directly into the Turing Pi switch.
Through the consumer router:
ptp4l: master offset -16580 s2 path delay 469541
ptp4l: master offset -178615 s2 path delay 485331
ptp4l: master offset 246261 s2 path delay 485331
Path delay around 480 microseconds, offsets swinging +/-700 microseconds.
Direct to the Turing Pi switch:
ptp4l: master offset -13 s2 path delay 2615
ptp4l: master offset 127 s2 path delay 2607
ptp4l: master offset -14 s2 path delay 2615
Path delay around 2.6 microseconds, offsets within +/-130 nanoseconds.
A 180x improvement in path delay just from removing the consumer router. Consumer routers buffer packets through software queues with variable and unpredictable delay. That variability goes directly into the PTP path delay calculation and degrades the offset estimate. The Turing Pi switch passes traffic at hardware speed with consistent delay.
I disconnected the GPS antenna for 10 hours overnight. Without GPS input, SatPulse goes into holdover and the grandmaster runs on the CM4’s local crystal reference alone. clockClass changes from 6 (GPS locked) to 52 (holdover).
The GM clock drifted to a maximum error of around 15ms over 10 hours. The more interesting result was on the clients:
#* PTP 0 0 377 0 +4ns[ +26ns] +/- 291ns <- RK1 after 10 hours holdover
#* PTP 0 0 110 4 -118ns[ -168ns] +/- 205ns <- k8s1 after 10 hours holdover
Both clients stayed on PTP the entire time. Neither fell back to WAN NTP. The crystal reference held well enough that chrony never saw a reason to abandon the PTP source.
For my use case this is sufficient. If you need guaranteed holdover accuracy across extended outages or power loss, you need hardware with a TCXO or OCXO, none of which are present in this build.
I set up Prometheus and Grafana to capture data from all three machines. The stack includes node_exporter and chrony_exporter on the grandmaster and both clients, plus a custom satpulse exporter on the grandmaster that parses journald output and exposes PHC offset, sync status, clockClass, and missing sample counts as Prometheus metrics.
Key metrics to watch:
satpulse_sync_status
, 1=GPS locked, 0=holdover
satpulse_clock_class
, 6=GPS locked, 52=holdover
satpulse_offset_max_nanoseconds
, worst-case PHC offset from GPS in the last 30s window
chrony_tracking_last_offset_seconds
, system clock offset from reference
node_timex_sync_status
, kernel sync flag, drops to 0 during outages
Light travels approximately 30 centimeters per nanosecond.
The grandmaster PHC is within 7 nanoseconds of GPS truth on average. That is the time it takes light to travel about 2 meters.
The CM4 client system clock is within 3-10 nanoseconds of GPS truth. That is the time it takes light to travel across a small room.
The finished grandmaster fits in a Waveshare CM4-IO-BOARD-CASE-A aluminum enclosure. The SMA antenna connector exits through the top panel. Five Dupont wires connect the SR1723U10 GPS module to the 40-pin header and J2 header inside.
The case lid holds the fan and has a cutout for the SMA bulkhead connector. The Dupont wires run from the GPS module on the board up through the case to the J2 header.
The GPS module sits near the 40-pin header on the upper left. The green wire carrying the PPS signal runs to the J2 header near the Ethernet jack. The PCF85063A RTC battery holder is visible on the lower left with a CR2032 installed.
The SMA bulkhead on the top panel connects to the Bingfu active GPS antenna via a JEFA Tech U.FL to SMA pigtail routed internally. The enclosure measures 160mm x 100mm x 40mm.
Most knowledge and/or memory management systems are primarily concerned with storing, maybe even organizing , information.
Facts is primarily concerned with deciding what information should become knowledge, and when, and how. It's as much about multi-player consensus as it is about fast capture and recall. The CLI is a zero-config, fast, malleable substrate for managing trusted knowledge. You start by initializing the system.
The basic unit is a proposition: a statement that can be evaluated as either true or false.
That proposition is not automatically a fact.
Someone, or something, still has to decide on it:
01a00-9ef94 pending, update pending Policy v0 [accept or reject]
$ fact accept 01a00-9ef94
Now it can participate in the ledger's accepted knowledge.
That small distinction is very powerful and enables multi-player curation.
A wiki says:
Someone wrote this.
Facts can say:
1. Someone proposed this.
2. These actors considered it.
3. This revision was accepted.
4. This is the version currently treated as fact.
That distinction matters for humans. It matters even more for AI agents. It enables transparency, tracing, and accountability.
Think of Facts as applying some of the useful properties of Git to knowledge.
You can propose something:
$ fact propose architecture.md --decision accept
Later, someone learns that it is incomplete:
$ fact revise 019fa-b6e41 architecture-v2.md
The old fact does not disappear just because somebody edited some text.
You can inspect what happened:
You'll see something like:
...
Revisions
01a00-2181f pending 2026-08-17T06:18:33.162Z ...
019fa-b6e41 accepted 2026-08-17T06:15:49.427Z ...
That's an important distinction.
This knowledge/memory system still has an accepted fact:
Revision 019fa-b6e41
It also has a proposed replacement:
Revision 01a00-2181f
The replacement does not become effective merely because it is newer.
It has to earn that status.
$ fact accept 01a00-2181f
Now the newer revision can become effective.
The history remains available:
$ fact history 01a00-2181f
Conceptually:
- proposed
- accepted
- revised
- commented
- accepted
- archived
- restored
- revised
- ...
Knowledge changes over time. The history explain how and why.
Consider a company wiki.
Someone adds: "Customers on the Enterprise plan receive 24-hour support."
It is immediately searchable.
That is useful if the statement is correct.
It's less useful if someone misunderstood the policy.
With Facts:
$ fact import wiki/enterprise-support-research.md
The system can retain the proposition without treating it as accepted knowledge.
A reviewer sees it, they investigate, and reject it:
$ fact reject 019fb-12a87
The proposition remains part of history. It simply never became a fact. Which gives you a useful distinction, i.e., "We considered this" is not the same thing as "We believe this is true."
Facts does not fundamentally care whether the actor is a human, an agent, a service, or an organization.
The following creates an operating environment for a registered actor with public/private signing key pair and permissions to participate.
$ fact as "Codex" --alias codex --home .actors/codex --participate --type agent
The agent,
codex
, can now propose:
$ FACT_HOME=.actors/codex fact propose runbook.md
The human can give themselves a nice name too:
$ fact as "Al Newkirk" --alias alnewkirk --self --type human
Another actor can comment:
$ fact comment 019fb-12a87 \
--message "This contradicts the production runbook."
Another can reject:
$ fact reject 019fb-12a87
Or several participants can be involved:
$ fact invite 019fb-12a87 alice $ fact invite 019fb-12a87 reviewer-agent $ fact invite 019fb-12a87 security-agent
The important concept is not "human approval."
It is controlled knowledge formation.
Humans can supervise agents.
Agents can supervise agents.
Humans and agents can participate together.
A common agent-memory loop looks roughly like this:
Agent observes something
|
v
Write to memory
|
v
Retrieve it later
|
v
Treat it as context
The danger is obvious once you look at it this way.
An agent makes one bad inference:
Customer X never wants an email notifications.
The agent stores it.
A future agent retrieves it.
Now a mistake gets committed to memory as truth. This problem can be compounded if that memory is shared causing a catastrophic failure.
Facts lets you put a decision boundary in the middle.
An agent can propose:
$ fact propose --message "Customer notification preference ..."
But a future agent querying accepted knowledge does not have to see it yet.
And human could approve it:
$ fact accept 019fb-6c222
Or another agent could review it first.
Or multiple participants could be required.
The workflow becomes:
observe
|
v
propose
|
v
review / deliberate
|
v
accept
|
v
recall as trusted knowledge
The agent is still free to learn and propose new information and revisions.
Consider This: Two agents using a shared ledger to check each other's work you now have a dynamic autonomous in-flight CI process.
A common AI architecture looks like this:
Slack
Drive
Wiki
Tickets
Email
Databases
Meeting transcripts
Documentation
|
v
"Agent, go figure it out."
The agent repeatedly has to determine:
- Which document is current?
- Which statement was superseded?
- Was this proposal ever approved?
- Was this just somebody's opinion?
- Which of these three policies is authoritative?
Facts gives you another layer.
Raw information can stay where it is.
Curated knowledge can live here:
$ fact find "production database access"
You might get:
019f91-a2f91 Production database access requires VPN
019fa8-3cfa8 Production credentials rotate every 30 days
019fa9-92fa9 Production writes require an approved change
Now the agent starts with three accepted facts instead of 4,000 search results.
Tagging helps actors organize and narrow vector search results even further.
$ fact find "production database access" --tag network --tag policy
If it needs supporting information, it can go fishing afterward.
The default workflow becomes:
1. facts first
2. raw information second
Instead of:
1. search everything
2. hope retrieval found the right answer
Start a ledger:
Record a preference:
$ fact import my-preferences.md --decision accept
Find it later:
$ fact find "writing style"
Revise it:
$ fact revise 019fb-ce177
Inspect it:
The result is more structured than a notes folder without becoming a heavyweight knowledge system.
Someone proposes:
$ fact propose deployment-policy.md
The team sees:
A participant comments:
$ fact comment 019fc-22d91 \
--message "Can we clarify whether emergency deploys are exempt?"
The proposition is revised:
$ fact revise 019fc-22d91 deployment-policy-v2.md
Then accepted:
$ fact accept 019fc-22d91
Six months later:
$ fact history 019fc-22d91
Now you can answer both: What is the policy, and how did we get here?
Small engineering conclusions frequently disappear into chat.
Facts can turn them into reusable knowledge.
$ cat <<'EOF' | fact propose - --decision accept # Invoice numbering The billing service is the sole authority for invoice numbers. EOF
Later:
$ fact find "invoice number ownership"
Another:
$ cat <<'EOF' | fact propose - --decision accept # Public identifiers Public API identifiers must not expose internal database IDs. EOF
An engineering agent working on billing can retrieve both.
It does not need the entire architecture wiki.
An incident produces a lesson:
$ cat <<'EOF' | fact propose - # Restarting the queue workers Queue workers must be drained before restarting the scheduler. EOF
An operations engineer reviews it:
$ fact accept 019fc-812a1
During the next incident:
$ fact find "restart queue scheduler"
The incident gets resolved much quicker and could possibly be automated with AI agents with access to the same incident response ledger.
The reusable lesson becomes knowledge.
A product manager proposes:
$ fact propose requirement.md
An engineer comments:
$ fact comment 019fd-1233 \
--message "This requires the billing API changes first."
A designer comments:
$ fact comment 019fd-12331 \
--message "The empty state is unspecified."
The requirement is revised:
$ fact revise 019fd-12331
Then accepted:
$ fact accept 019fd-12331
Now an implementation agent can query accepted requirements rather than parsing every planning discussion.
Suppose an agent analyzes customer tickets overnight.
It proposes:
$ for file in agent-findings/*.md; do fact import $file; done
The next morning:
A human accepts three findings and rejects two.
The agent discovered information, decide what to do with the rejected propositions, and continues.
The interesting thing about the Facts system is that the same primitives can support very different systems.
+----------------+
Human A -->| |
Human B -->| Single |
Agent A -->| Ledger |
Agent B -->| |
+----------------+
Everybody works from one shared knowledge domain.
Typical commands:
$ fact pull $ fact pending $ fact propose discovery.md $ fact push
Useful for a small team with broadly shared knowledge.
+--> engineering
|
Agents/Humans --+--> product
|
+--> operations
|
+--> security
Switch domains:
$ fact use engineering $ fact find "authentication" $ fact use security $ fact find "authentication"
Same query.
Different curated knowledge boundary.
An agent can be granted only the knowledge domains it needs.
Create a dedicated ledger:
$ fact init project-atlas
Capture:
$ fact propose requirements.md --decision accept $ fact propose constraints.md --decision accept $ fact propose architecture.md
During development:
$ fact find "database"
$ fact pending
$ fact revisions 019fe-22311
When the project ends, the ledger remains as durable project memory.
The next team does not have to reconstruct the project from Slack history.
A ledger can function like an inbox.
Agent A -----+
|
Agent B -----+--> shared ledger --> Worker Agent
|
Human -------+
Submit work:
$ fact propose investigate-cache-misses.md
Worker:
$ fact pending
$ fact echo 019fe-8bb91
The worker acts, then records an outcome:
$ fact revise 019fe-8bb91 result.md $ fact accept 019fe-8bb91
A proposition becomes both communication and durable record.
+--> researcher ledger
Human/Agents --+--> security ledger
+--> coding-agent ledger
+--> planner ledger
An orchestrator switches context:
$ fact use researcher $ fact propose research-request.md $ fact use security $ fact propose review-request.md $ fact use coding-agent $ fact propose implementation-request.md
Each agent sees only its own work and memory.
The ledger becomes:
inbox + memory + history
without Facts needing to define an agent framework.
Research Agent
|
v
proposition
|
+------> Verification Agent
|
+------> Policy Agent
|
+------> Human Reviewer
|
v
accepted
For example:
$ fact propose market-observation.md
Invite reviewers:
$ fact invite 019ff129-a71 verification-agent $ fact invite 019ff129-a71 policy-agent $ fact invite 019ff129-a71 alice
Their decisions determine whether the observation graduates into shared memory.
Suppose no single AI should be trusted to update critical memory.
Agent A: accept
/
Proposal ----- Agent B: accept
\
Agent C: reject
The ledger records each position.
Consensus policy determines whether the proposition becomes effective.
That is very different from:
Agent A wrote something to memory, therefore it is memory.
You can deliberately combine different models or agents with different tools so that their failure modes are less correlated.
An agent has permission to propose but not decide.
Agent:
propose
comment
Human:
accept
reject
revise
Agent:
$ fact propose learned-preference.md
Human:
$ fact pending $ fact open 019ff-872cc $ fact accept 0119ff-872cc
The agent learns continuously.
The human controls what graduates into durable memory.
Invert the previous model.
Routine facts are handled by agents.
Humans appear only when something is contested.
Proposal
|
+--> Reviewer Agent A
|
+--> Reviewer Agent B
|
disagreement?
/ \
no yes
| |
settle human
The human's inbox can effectively become:
$ fact conflicts $ fact pending
Humans manage exceptions instead of reviewing every memory update.
Facts ledgers can move between machines.
Configure a remote:
$ fact remote add origin https://facts.example.com/team
Synchronize:
Work locally:
$ fact propose architecture-note.md $ fact accept 019ff-a8211
Publish:
Another participant:
$ fact pull
$ fact find "architecture"
The useful property is not simply synchronization.
It is that the knowledge is not trapped inside the application that currently consumes it.
Suppose today you use:
Agent A
Six months later:
Agent A is retired.
Agent B replaces it.
If Agent A owns its memory, you have a migration project.
If the organization owns a Facts ledger:
Agent A --> Facts <-- Agent B
the replacement starts with the same accepted organizational knowledge.
Models change.
Frameworks change.
Vendors change.
Agents come and go.
The ledger remains.
At first glance, Facts is intentionally small.
You can:
- fact propose - fact revise - fact comment - fact accept - fact reject - fact find - fact pull - fact push
But those primitives compose into:
That is the point.
Facts does not try to predict the application.
It provides the substrate.
And underneath all of those topologies is the same simple distinction:
Someone said it.
is information.
We accepted it.
is a fact.
Information accumulates. Knowledge is curated.
I thaca has one hell of an opening: Penelope, a disillusioned environmental lawyer, stops her car at a petrol station. But upon returning, she discovers something shocking: the unconscious body of an oil company CEO in her boot, his hands hog-tied together, dangling limply towards the vehicle’s number plate.
This tantalising eco-thriller setup is a long way off the typical knights-and-castles fantasy fare of most role-playing games, more How To Blow Up a Pipeline than Dragon Quest . But road-trip RPG Ithaca arrives via refreshingly original developer the Pixel Hunt, a French outfit that specialises in what it calls “reality-inspired” games. The studio has made a number of arresting works in recent years. One was about a Syrian refugee couple making a perilous migrant journey across Europe; another tackled the life-shattering effects of child sexual abuse. The climate crisis was always in the back of Pixel Hunt founder and Ithaca writer Florent Maurin’s mind. After all, he says, it is the uber problem of our day, the problem that touches all others. It was just a case of when, and, more importantly: “What do we say about it?”
Then, one summer afternoon a few years ago, while out for lunch in the French medieval town of Cluny, Maurin was posed a question by his wife: “How, considering all we know scientifically about the climate crisis, do we explain to our daughters that we didn’t spend every weekend marching, protesting and demanding the powers that be to take meaningful action?” Maurin was silenced. “I had no fucking clue,” he says. “I had no answer to that question.”
Maurin’s eventual response was to tackle the subject the way he knew best – by making a video game. In Ithaca, you drive along a procedurally generated road, soaking up vistas of huge windfarms, mountains passes and gorges filled with clean blue water. You talk to family and friends, with the decisions you make affecting the direction you take on the asphalt highway and the branching storyline. Slowly, you uncover what the radical climate group Penelope is now part of, the Earth Protection Association, is really planning to do with the oil exec in her car’s trunk, this nefarious avatar of petro-capitalism. Does he walk free or pay for his Earth-heating sins? Or does something else happen entirely?
The prickly, ethical questions that Ithaca raises are influenced by the real world’s growing upheaval. Maurin has been struck by what he sees as a sea change in attitudes towards political violence. He cites the reactions to Luigi Mangione’s alleged killing of UnitedHealthcare CEO Brian Thompson and the assassination of rightwing activist Charlie Kirk. “You might have expected people to say this is not something that they, or anybody, should do, but actually, a sizable part of the population in the US cheered and was like: ‘This is right. It’s deserved.’”
The road Penelope ventures down is something of a metaphor for “where we are all heading”, says Maurin. As the mythic home of Odysseus, Ithaca, Penelope’s ultimate destination, was chosen for its allegorical meaning: Maurin says it offers an opportunity to think through ideas of home in a broader, almost planetary sense. He sees the current moment as one in which “we’re trying to discover where we want to go collectively as a species”.
It is an urgent question, dovetailing with Ithaca’s high-octane story and what Maurin sees around him on a daily basis in Burgundy, central France. When he arrived, 10 years ago, it was verdant, an almost staggeringly green place. “You felt as if water was everywhere in the landscape,” he says. Now, drought arrives every summer, killing any trees ill-suited to such high temperatures and a lack of water. Hillsides are littered with their grey and yellow arboreal remains. “Every time, I look out of the window and I think, ‘We have to talk about this very fast,’” says Maurin. “Because things are changing way too quickly for nature to be able to adapt.”
ON JULY 21, 2026, New York City Mayor Zohran Mamdani posted a video titled, “Benjamin Netanyahu is a war criminal.” The video quickly went viral across social media platforms and drew praise for its unapologetic call for accountability — the first by a highly recognized US politician — after years of Israel’s ethnic cleansing and genocidal actions in Gaza. Mamdani also acknowledged that he lacked the legal authority to fulfill a highly publicized campaign promise to arrest the Israeli Prime Minister if he entered New York.
It is unclear why Mamdani thought the New York Police Department (NYPD) would have the authority to make such an arrest, and why he repeated that promise on the campaign trail when other marquee campaign promises like a four-year rent freeze, free childcare, and fast and free buses were strategically selected as achievable reforms in the current legal landscape. Whether the promise stemmed from naivete, a political calculation, or some combination of the two is unknowable.
It was always unlikely that New York law would give the City the power to enforce a warrant from the International Criminal Court, but the promise did generate a news cycle that ended with Mamdani’s call for the federal government to take action and arrest Israel’s Prime Minister. While the Trump administration is unlikely to arrest its international ally, Mamdani’s call will hang over the 2028 Democratic Presidential primary as an energized party base continues to make clear that public sentiment is moving away from Israel and towards solidarity with the Palestinian people.
The promise to arrest Netanyahu had offered a tempting rebuttal to then-NYC Mayor Eric Adams and the political establishment. Early in his mayoral campaign in an interview with Chapo Trap House, Mamdani railed against Adams’ enthusiasm for using the NYPD to break up pro-Palestinian student encampments on university campuses. Mamdani called on the government, specifically the NYPD, to stop violating the First Amendment rights of students protesting in solidarity with the Palestinian people and instead arrest a head of government with an outstanding arrest warrant for war crimes. Such a framework fits comfortably in the left-wing critique that our legal system is only unleashed on working-class people while the rich and powerful dodge accountability for their misdeeds.
Based upon his past activism and statements, Zohran Mamdani’s desire to hold Netanyahu accountable for the crimes of the Israeli government is genuine. As a college student, Mamdani was the co-founder of Bowdoin college’s chapter of Students for Justice in Palestine (SJP). He has said that the struggle for Palestinian liberation was his entry point into US politics, an unlikely origin for an elected official.
As Mayor Mamdani seeks to build solidarity with the Palestinian people and dismantle Israeli apartheid, he should look to the example set by another activist-turned-mayor, Marion Barry. The District of Columbia’s “Mayor for Life” effectively used his platform to raise awareness of the anti-apartheid struggle in South Africa, and used his office to support organizing against apartheid within the DC government itself. Barry’s efforts sought to mobilize popular opposition to the Reagan and Bush Administration continuation of the US-South African military and economic relationship through a policy of “constructive engagement”. 1
Mamdani himself has noted the strong connection between the Palestinian and South African freedom struggles, including Nelson Mandela’s own support for the Palestinian cause. 2 Mamdani cited a June 1990 town hall interview with Nelson Mandela at the City University of New York, where Mandela noted that leaders like Palestinian President and Palestinian Liberation Organization (PLO) chairman Yasser Arafat supported the South African struggle “to the hilt.” Mandela would further elaborate to rapturous applause that his African National Congress party identified with the PLO because “just like ourselves, they are fighting for the right of self-determination.”
Marion Barry, born to sharecroppers in the Mississippi Delta, was elected the first chairperson of the Student Nonviolent Coordinating Committee (SNCC) in May 1960. Political allies of Barry noted the deep impact of SNCC organizing on his campaign and governing mindset. Many SNCC members would later staff his electoral campaigns and administrations. SNCC’s efforts to organize and register poor, disenfranchised Black farmworkers to vote in the South informed his notion that campaigns should widen the electorate and bring poor and working class Washingtonians into the political process. Barry’s landmark summer youth jobs program stemmed from his own experience as a young Black person in the South who could not find employment in the formal economy and was forced to collect bottles and old newspapers to make money in the summer.
The Mamdani connection to SNCC runs directly through Zohran’s own father, anti-colonial academic Mahmoud Mamdani. Mahmoud was attending the University of Pittsburgh as a Ugandan exchange student on scholarship when he was recruited to join a SNCC action in Montgomery, Alabama. He was arrested for his participation in the civil rights movement, which prompted intervention by the Ugandan ambassador to the United States to secure his release. The elder Mamdani later noted that his identity as an Asian person in Uganda made him more sensitive to the racialization people can face in their own society, and he felt an obligation to join in the struggle against Jim Crow while studying in the US.
Elected to the first of three consecutive terms as mayor in 1978 (and a fourth non-consecutive term in 1994), Barry had to strike a balance between his reputation as a national activist and young political rising star, and his role as a municipal official elected to solve local issues. When asked if he would travel across the country to campaign for DC statehood and autonomy, Marion Barry replied , “I’m not interested in traveling. I’ve traveled enough in my life. I want to stay right here and work for the citizens of the District of Columbia.” 3, 4 At a June 2025 New York City Democratic mayoral debate, Zohran Mamdani echoed this idea when he told the moderators he preferred to stay in New York during his term, rather than commit to any foreign trips as Mayor. That offered a sharp contrast to Mamdani’s opponents, many of whom enthusiastically volunteered that if elected mayor, they would visit Israel as soon as possible.
Despite both Mamdani and Barry’s pledges to keep their focuses local, their activist backgrounds and progressive bases became apparent on international issues. Support for Palestine is not surprising in New York City, considering the growing political engagement of South Asian and Muslim communities, and the increased salience of the issue for younger voters. November 2025 exit polls indicated that two-thirds of NYC voters said their preferred candidate’s position on Israel (a slanted framing) influenced their vote, and Mamdani carried those voters 49% to Andrew Cuomo’s 43%.
Likewise, in the then-majority Black capital city of the United States, Marion Barry found an electorate that was supportive of the international movement to end apartheid in South Africa. Several years into his tenure as mayor, Barry began using the bully pulpit of the Mayor’s office to speak out against apartheid, including at rallies outside the South African embassy. Beginning in November 1984, protesters would gather outside the embassy to demonstrate, while individuals crossed onto embassy property to seek arrest. Police arrested thousands of demonstrators over several months of protests, including on January 15, 1985 when Effi Barry, the First Lady of the District of Columbia, and United Auto Workers President Owen Bieber were among the 17 protestors arrested. Commenting on his wife’s arrest, Mayor Barry said that he didn’t want his then four and half year old son to grow up “and know that his mother and his father didn't do all they could to crush this evil system.”
In another use of his public office, Barry spoke before a mock funeral procession to the State Department in August 1985, railing against what he deemed the failed federal policy of “constructive engagement” towards South Africa, which upheld the “indefensible system of Apartheid.” Barry also said:
“As mayor of this great city, we encourage all of you and those of us who advocate against the white supremacist South Africa to continue to march. Some of us want to go even further than that. We think there ought to be an arms embargo on the sale of any arms from this country to the government of South Africa. And if that does not work, we ought to have a naval blockade of South Africa to make sure that they can’t continue this.
Our own city of Washington has very strong divestiture law [that] makes sure that no city money is invested in any bank, in any security, that’s doing business in South Africa. We’ve already taken $100 million out of our pension fund, we’ll take it all out if necessary. So we are here, not only as mayor of our great city, but as President of the National Conference of Black Mayors representing some 290 Black mayors in America growing in unity and growing in number.”
Mayor Barry then shared the story of a South African intern in his office who took deep pride in the American show of solidarity with the South African people. He closed with the call, “Down with apartheid, up with freedom, justice, and equality.”
At another rally in January 1986, Mayor Barry told a crowd outside the South African embassy that he had sent legislation to the DC Council to rename a stretch of road outside the Embassy “Nelson and Winnie Mandela Avenue,” after the two high-profile ANC prisoners. While that change was not adopted, current Mayor Muriel Bowser would later use a similar tactic by renaming the street outside the Saudi embassy after slain journalist Jamal Khashoggi and renaming the stretch of 16th Street NW just north of the White House “Black Lives Matters Plaza.” In his first year in office, Mamdani used his official city social media account to post a video titled, “ A visit with a Palestinian Nakba survivor ” on May 15, observed as Nakba Day , likewise showing the ability of elected officials to use their platforms to challenge prevailing narratives in political and historical debates.
The most impressive display of the mayoral power to organize against apartheid took place on April 4, 1985, when Marion Barry spoke outside the South African embassy in front of a crowd of 4,000 city employees on what was dubbed “ DC Government Employees Day Against Apartheid .” With the mayor’s approval, staff within Barry’s administration had organized for weeks by talking to and flyering their coworkers on their lunch breaks to recruit coworkers to take two hours of leave that day - the 17th anniversary of Martin Luther King’s assassination. Police reported they arrested at least 78 District employees that day. At the rally, Barry declared, "[t]he South African government is going to be under siege at home and abroad until it lets its people be free."
This action’s scale and success can in part be attributed to Barry’s efforts to diversify and integrate historically excluded people into the District government during his 1978 campaign and while in government. Barry, like Zohran Mamdani, focused on less likely or first-time voters to power his winning coalition, much to the shock of pollsters and political analysts. Barry appointed his allies from SNCC and other Black Power organizations to high-level positions, as well as important seats on District boards and commissions. Barry intentionally sought to hire women, LGBT folks, and residents from all eight wards of the city into a new home rule government that had been disproportionately staffed by professional class white men. In New York, Zohran Mamdani leads a city government of over 300,000 employees. 5 He has a unique opportunity to protect and legitimize organizing by employees who might have been fearful to speak up about Palestine in prior administrations. Those workers are best positioned to educate their coworkers about the historic exclusion of Palestinian people from the claims of universal human rights.
Ultimately, a sizable number of Republicans in both the House and Senate would join with Democrats to override President Reagan’s veto of the 1986 Comprehensive Anti-Apartheid Act and impose sanctions on South Africa until the apartheid system was dismantled. Such a reversal of US foreign policy, as well as the bipartisan reversal of a Presidential veto, would not have happened without years of popular organizing and pressure, including locally in Washington, DC.
In 1990, the trajectory of Marion Barry’s career and the fight against South African apartheid took dramatic shifts. Amidst growing fiscal challenges for the District of Columbia and the heavy carceral response during the so-called “War on Drugs,” the FBI arrested Barry during a sting operation on January 18th, after an ex-girlfriend working as an informant convinced the mayor to use crack cocaine while being secretly recorded. That arrest would lead to his departure from the 1990 Mayoral election. He was sentenced to six months in prison in October and faced the only electoral loss of his career in November when mounting an independent bid for an at-large council seat.
After serving his sentence, Barry would return to District politics in 1992 as the Ward 8 Councilmember and would win his fourth and final term as Mayor in 1994, the same year Democrats would lose control of the House of Representative for the first time since 1955. The politics of neoliberal austerity came to dominate both federal and District policy. 6 First under the Congressionally-imposed Financial Control Board in 1995 and then under subsequent mayoral administrations. 7
After decades of domestic and international pressure, including the global campaign of divestment and sanctions and a shocking military defeat at the hands of Angolan and Cuban forces at the Battle of Cuito Cuanavale , the apartheid government of South Africa released Nelson Mandela from custody on February 11, 1990. After his release, Mandela received a hero’s welcome in the United States. In June, after a charter flight from South Africa to New York on the now defunct Trump Shuttle , Effi Barry was part of the 10-member delegation that welcomed Mandela to the DC area at the airport while her husband dealt with his on-going trial. 8 While forced to miss many of Mandela’s events in DC, including his address to Congress, Barry received raucous applause when introduced at the DC Convention Center before a public address by Mandela.
By 1995, white rule had been dismantled, sanctions had been lifted, and Nelson Mandela served as the first Black President of South Africa after apartheid. 9 Marion Barry would also make his return, but the intensifying attacks on DC home rule continue to the present day. 10 Marion Barry’s political comebacks and his service as the Ward 8 councilmember until his death show his enduring impact, influence, and popularity. Many other elected officials had scandals over the course of their careers, but no scandals in white communities had such an outsized impact on their residents. The federal investigative resources spent to convict Barry never turned up any evidence that he personally sought financial gain, but the lurid media narratives of personal drug use and extramarital sex were weaponized against Barry and ultimately the entire notion of self-governance for Black Washingtonians.
New York City is home to the United Nations, along with diplomatic and commercial nodes, and is an even more international city than Washington, DC was at the height of the anti-Apartheid movement. Mayor Mamdani will have numerous opportunities to use his bully pulpit to call out war crimes, illegal occupation and land theft, settlements, and other racist actions by the Israeli government against the Palestinian people. As public opinion continues to shift, the mayor should utilize these opportunities to uplift organizing in New York City in order to continue to shift the Overton window on US-Israel policy and challenge anti-Palestinian tropes and narratives.
Mayor Mamdani has skyrocketed up the list of most recognized political figures, while so far maintaining relatively high favorability. His promise to arrest Netanyahu catalyzed public support for Palestinian liberation into a tangible policy for federal officials. A July 28th YouGov poll shows Americans support arresting Netanyahu 49%–27%, an astounding 22% margin. Democratic voters support arresting Netanyahu 68%–11%, an overwhelming 57% point margin. It would be hard to imagine this question even being asked of voters without the Mayor of New York (or some equivalent figure) calling the question in such a public manner.
Marion Barry played a major leadership role in the National Conference of Black Mayors and used that forum to discuss human rights of importance to his political base. Likewise, Zohran Mamdani can use his position as “America’s Mayor” and one of the leading critics of the Trump administration to build networks of support with other mayors and local officials who are aligned with him on Palestine and other movements for social justice. 11 As Mayor Mamdani and others find messages that resonate with voters, it will become progressively easier for other elected officials to echo those sentiments.
Mayor Mamdani, echoing Barry’s call for District government divestiture from companies doing business in South Africa, supports the 2006 call of Palestinian civil society for Boycott, Divestment, and Sanctions against Israel until the core demands of an end to illegal occupation, equal rights, and a right of return for the Palestinian people are honored. Building off his “ Not On Our Dime” Act legislation introduced while in the New York Assembly, Mamdani could support restrictions on firms or nonprofits within New York City that do business in Israel, especially those operating in the occupied territories or contractors supplying military technologies.
The Mayor of New York, like the Mayor of DC in the 1980s, can also offer effective rebuttals to the positions of the White House and State Department and repeated by the media. Looking towards the 2028 election, Zohran should build upon his viral call for the arrest of Netanyahu and continue to use his platform and favorability within the Democratic party to press the candidates seeking the Presidential nomination (a list likely to include his ally Rep. Alexandria Ocasio-Cortez) and other federal offices to speak out about the concrete need for shifts in United States policy towards Israel and Palestine.
This is not to say these shifts will happen automatically, nor will they be quick, but Zohran Mamdani has been working on these issues since he was a student organizer, then as a political candidate and member of the New York Assembly. As the decades long-struggle against South African Apartheid shows, immense, seemingly impossible change is possible with sustained organizing - and that local governments can represent an important site of struggle for global movements for liberation.
Marion Barry’s impacts as a civil rights organizer and his early reform-oriented tenure as mayor deserve acclaim and study today. As a new generation of activist mayors such as Zohran Mamdani and Janeese Lewis George - both members of Democratic Socialists of America - take office in major cities across the United States, Marion Barry offers an example of how local leaders can use their offices to boost global movements of solidarity and justice.
1 - The Reagan policy of “constructive engagement” bears similarity to the Biden Administration’s “Bear Hug Strategy” with Netanyahu’s government after October 7th.
2 - Mamdani shared in that talk that as a young person growing up in South Africa, Mandela “was the first President he ever knew.”
3 - "Marion Barry, Mayor-Elect: In Search of a National Reputation: Marion Barry, Mayor-Elect: Questing After a National Reputation." The Washington Post (1974-), Dec 14, 1978. ( Accessed August 3, 2026 )
4 - Barry would take a 19 day trip to Africa in his first year in office, but reiterated after his visit that his “priority is Washington, DC.” He was also asked on the trip about President Jimmy Carter’s firing of UN Ambassador Andrew Young after a secret meeting with PLO officials. Barry noted that while it might hurt Carter with Black voters in the short run, ultimately there were not many options on the Republican side to support.
5 - Returning to the previous 2025 exit polls and views on Israel-Palestine, organizers for Palestinian solidarity have a base of over 500,000 voters who have been politicized around the issue by the Mamdani. This presents an organizing opportunity that has likely grown with the success of other DSA candidates in 2026.
6 - While serving as the primary employment center for one of the nation’s largest metropolitan areas , middle class white flight and then Black flight substantially undercut the District’s tax base and undermined the ability of the District to pursue more redistributive social policies. Unlike states with split metro area economies, the District gets no income tax reciprocity for commuters who live outside of DC. The dependence on property taxes, where government and nonprofit properties are untaxed, further exacerbates inequality. This creates a fiscal incentive for the District government to promote gentrification of existing homes and real estate, as private development is one of the most certain streams of potential revenue (and pushing poorer residents out of the city can decrease spending on public services).
7 - Angela Davis is quoted as saying to Barry in a crowded room during this period, “We stand with the Barry of 1968, not the one of 1998.”
8 - The shuttle was not a gesture of generosity from the future President, rather Mandela’s team reportedly paid $130,000 for the flight .
9 - It bears consideration that neither Zohran Mamdani (b. 1991) nor Janeese Lewis George (b. 1988) were alive to experience these lessons directly and that the youthful movements sweeping politics including DSA, have a lot to learn from these local struggles to build solidarity around the world to dismantle white supremacy. Many radical movements identified self-determination for Black South Africans then, and Palestinians today as a key step in dismantling racial capitalism.
10 - Barry and allies did not shy away from comparisons between South Africa and DC in this period.
11 - Anecdotally, as a young elected official the impact of Zohran Mamdani on my rhetoric and the rhetoric of my peers is hard to understate. In many informal conversations, elected officials, especially younger progressive officials, express a deep admiration for Mamdani’s communication skills and style.
Frankie Fritz is a Metro DC Democratic Socialists of America organizer and socialist in office who currently serves on the Greenbelt City Council. Fritz’s first campaign was as an undergraduate student organizing for fossil fuel divestment, a struggle inspired by the campaign to divest from companies profiting from South African Apartheid.
The monthly Washington Socialist informs readers about perennial and evolving aspects of democratic socialism. Each issue includes essays, briefs, and articles covering topics oriented to specific local campaigns and broader socialist analysis. Each issue of the Washington Socialist is a produced by members of the Metro DC Democratic Socialists of America , although contributors from the DMV's political left should always feel welcome to contribute!
A new book by longtime Bay Area journalist Gil Durán investigates “tech fascism” and its mounting influence on U.S. politics. The Nerd Reich: Silicon Valley Fascism and the War on Democracy follows the rise of Vice President JD Vance, the former venture capitalist whose “outsider” campaign for Senate was bankrolled by the billionaire co-founder of PayPal and Palantir, Peter Thiel. Durán traces the ideological lineage of the current Trump administration back from Vance to Thiel and the far-right-wing monarchist Curtis Yarvin, whose “right-libertarian” political theory has long made the rounds among Silicon Valley elite. “These guys were never libertarians,” says Durán. “Now that they are the government, we see their true face: They’re fascists, and they’re authoritarians.”
Bay Area-based journalist and author.
Please check back later for full transcript.
The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
Debian LLM GR - Summary of the options
LLMs have finally made it to the ultimate stage of Debian’s governance processes, a General Resolution of all the project’s full governing members (DDs).
There are a lot of options on the ballot, and they all have a different structure and approach the question in a different way. It can be hard to see the wood for the trees. I have made a summary table to try to capture the main differences, both in effect, and sentiment.
Suspending briefly my attempt to be neutral:
Before voting, I encourage you to read the passionate rationales in options H and A, or at least the summary in my option C.
Few of the LLM defences in the discussion threads, and none of the LLM-positive proposals, provide answers to any of these profound ethical concerns, many of which ought individually to be a deal-breaker. Instead, these crucial questions are simply dismissed or even ignored.
Some will tell you we should “keep politics out of software” but as we can see in the world around us, software is political - now more than ever. Debian’s mission is a highly political one: developing a fully-free operating system, and defending its freeness as we do, is far from neutral!
And of course many of LLMs’ harms affect Debian directly.
| A | G | C | H | F | D | B | E | |
|---|---|---|---|---|---|---|---|---|
| LLM harms | Robusly discussed | Discussed | Robusly summarised | Robusly discussed; especially re climate | Summarised | Accepted as inevitable | Disregarded [1] | Ignored |
| Direct contributions of LLM-generated code | Forbidden | Forbidden | Strongly discouraged | Strongly discouraged | Discouraged | Permitted | Permitted | Permitted |
| Direct use of LLM output in communications (bugs, mailing lists, etc.) | Forbidden | Forbidden | Forbidden (with possible exceptions) | Strongly discouraged | Discouraged | Permitted | Permitted | Permitted |
| LLM use where LLM output does not end up in the code/message | Forbidden | No position, so permitted | Strongly discouraged | Strongly discouraged | Discouraged | Permitted | Permitted | Permitted |
| Disclosure of LLM use | LLM use forbidden | LLM use largely forbidden, no further disclosure requirement | Disclosure required | Disclosure encouraged | Disclosure encouraged | Disclosure required | Disclosure required | Undisclosed LLM use is OK |
| Use of LLMs by upstreams | Condemned | “Not recommended” | ||||||
| Positive statements about LLMs | “Here to stay” | Moderate | Strong |
I have tried to present the options in semantic order, with most LLM-negative proposals to the left, and the most LLM-positive to the right.
I have not quoted the one-line titles for the options. These have generally been provided by the proponents of each option, and, unfortunately, some of them are IMO quite misleading.
Note that, unfortunately, the voting software likes to assign numbers to options but also to preferences. Be mindful of this possible confusion when casting your vote. For clarity I quote only the option letters.
Some of the proposals acknowledge the uncertain legal status of LLM output. But all of them implicitly or explicitly assume that LLM output is or can be DFSG free. So none of the proposals forbid upstream projects with LLM-generated contents.
None of the proposals would require us to go back to pre-LLM versions of the upstream projects we use, and attempt to fork and maintain them. I very much think there is room in the world for people to try to do that, but I don’t think the Debian project can be that effort.
Given that the conclusions are the same in each case, whether the matter is discussed does not seem to me to be a significant difference. I have therefore not included a column for it.
My proposal has a specific paragraph (7) explicitly permitting teams to set a “no LLM” policy. The other proposals do not discuss this point specifically. During the discussion, it seemed that most participants agreed that even options which explicitly permit LLM use generally do not prevent a team from setting its own more restrictive LLM policy.
I have therefore not tabulated this aspect.
Few of the permissive texts are absolute or unconditional. To summarise I have necessarily left out some nuance.
So for example when an entry says “permitted”, that generally means “permitted with conditions which are believed by LLM users to be readily satisfiable” (for example, DFSG-compatibility - see above).
Proposal B does mention that there are “concerns” about LLM use. But it fails to make an explicit statement about whether these concerns are justified.
It then proceeds exactly as if they are not justified. IMO “disregarded” is a relatively mild term for such a rhetorical technique.
1984 not only brought us the Mac, it brought us an 8-bit app that made it easy to resist purchasing a Mac.
Dig, if you will, a picture.
It's November 1983 and throngs of people are engaged in literal fisticuffs, " dozens of people were injured, some even hospitalized ," to claim the last homely, pug-nosed Cabbage Patch Kid from the local Zayre's for Christmas shopping. In video footage, a store manager climbs a display, brandishing a baseball bat, trying to will order onto the chaos. The riots make the local, then national news, and eventually have a page on Wikipedia . It's a striking exclamation mark to punctuate 1983, with dark implications for the coming new year. Had we sunk so low?
Then, at the start of December, a Christmas miracle.
The hottest pop star in the world released one of the most influential music videos of all time on MTV. You know the one, where Michael turns into a werewolf, but then he's a zombie, and they do that dance, and the electronic beat "ur URRR adugga dugga dugga dugga dugga dug ur URRR adugga dugga dugga dugga dugga dug", and the Vincent Price laugh, and those crazy eyes at the end. The video for "Thriller" released simultaneous to the riots, and karmic balance was restored. Huzzah! Maybe 1984 wouldn't be so bad after all.
1984 wasn't just "not bad," it was radical.
I tried to find "cutest adorable chubby-cheeked preschoolers Thriller dance" but this will do.
Pop-culture nerds, in particular, would gorge themselves that year with Ghostbusters , The Terminator , Gremlins , Nightmare on Elm Street , Beverly Hills Cop , Revenge of the Nerds , Splash , Red Dawn , The NeverEnding Story , The Adventures of Buckaroo Banzai , Lynch's Dune , Karate Kid , Repo Man, This is Spinal Tap , Sixteen Candles , The Transformers toys and show, Teenage Mutant Ninja Turtles comic, Stop Making Sense concert film, Prince's Purple Rain , Do They Know It's Christmas? , Neuromancer , Elite , Karateka , King's Quest , the Soviet boycott of the Olympics , and the first MTV Video Music Awards. "Thriller" lost "Video of the Year" to "You Might Think", but did win "Best Choreography" and "Viewer's Choice."
Oh, and Apple aired a commercial during Super Bowl XVIII. You know the one, with the skinheads in grey, and the oppressive fascism, and the woman in red shorts, and the sledgehammer, and that electronic siren "BEE oooh.... BEE oooh", and "1984 won't be like 1984."
Apple introduced the Macintosh, a system Steve Jobs said was so easy to learn, "You can sit your grandmother in front of this computer and she’ll figure out how it works." That's an interesting way to describe a computer that internal documentation shows "we will not attempt to position the product in any way as a 'home' computer."
Meanwhile, that same year, the Apple II series sold another one million units while Apple was taking its third stab at obsoleting those machines. Having shrugged off the Apple III, then the Lisa, Apple II fans would be forgiven for looking at the Macintosh's anemic launch software library (essentially just MacWrite and MacPaint ) and shrugging in agreement with Clara Peller's 1984 catch-phrase, "Where's the beef?"
Tom Weishaar, writing for Open-Apple newsletter in July 1985 (a rough year for Apple), "For six years now Apple's management has been trying to build a computer better than the Apple II. In all these years, after all this money, only the Apple II has ever shown a profit. Buyers are looking for tools they can use. The Apple II has plenty of power to be useful."
VisiCalc was early, clear proof that innovation isn't measured in megahertz. By 1984, developers had a lot of expertise in making the most of limited hardware. Look at the strides made in six years of software research on the Atari 2600, for example. Practice makes perfect.
Rupert (later "Robert") Lissner heard the call. While the Macintosh was an interesting promise of a computing future that may or may not come to fruition, there were millions of Apple II owners with immediate needs. AppleWorks fulfilled those needs, growing a strong fanbase, spawning dedicated newsletters, lots of books, add-ons, and more. When Apple finally abandoned their 8-bit line, AppleWorks singlehandedly kept those machines productive for another decade .
My only experience with AppleWorks was in the early OS X days, and that version had nothing to do with what we're looking at today. That and this only share a name; the Apple II version holds a legacy.
Version 3 of AppleWorks was, according to a Compute! Magazine review, when the promise of "integration" finally paid off. Much of what AppleWorks 4 and 5 do, 3 can do with plug-ins. Version 3 is also when the program became Y2K compatible. It should be more than enough to develop a strong understanding of what made it such a favorite among the Apple II faithful.
Thus far, I've only looked at one other "integrated" software package: Pipedream , on the Acorn Archimedes. It put forth the hypothesis that word processing, spreadsheets, and databases are not three separate applications, they are one and the same, chemically.
While I was interested in its proposition, I was cool on its execution. It's an acquired taste. Where Pipedream blurred the line between applications, AppleWorks is very much three apps in one; a veritable turducken of productivity software.
Maybe I'm a little unfair with that analogy. A turducken is a forced alliance of three fowl under, let's call it "extreme duress." AppleWorks, on the other hand, is a joyful alliance, with each individual app supporting the other two. I'd argue the word processing benefits the most, but to categorize this as merely three applications which happen to ship in one box, would miss the turkey for the duck.
With 128KB of RAM and an entire suite of software loaded up, AppleWorks has 23KB remaining to do real work, as indicated by the memory counter in the bottom right of the screen. That's not a lot, but it's a bit dependent on how hardcore your work is.
For example, my pre-edited text for this blog post hit 9,000 words and 43KB; the database merge (I'll talk about later) hit 24KB. Both are substantial projects, so if you're just dashing off letters, giving publishers a chance to get in the ground floor of your burgeoning career as the author of legally distinct James Bond-alike, James Epoxy, you'll be fine. Chronicling Agent Epoxy's actual adventures will likely require you to scope documents chapter-by-chapter.
In the wake of the release of AppleWorks , a decent amount of third-party software was published to "add value" to this triple-threat. Using ad-hoc plug-in strategies (not originally part of the program proper), the already stuffed AppleWorks could become almost an operating system unto itself, hosting not just enhancements to the core functionality, but entirely new applications like screen savers and a MacPaint clone.
Other companies took a less invasive approach to enhancing AppleWorks , offering pre-built documents ready for mail merge, recipes, phone directories, and the like. I saw a reference to a fantasy football management suite called FantasyWorks , but couldn't find any disk images for it. I suspect there's an entire world of unarchived AppleWorks enhancements sitting around on forgotten floppies.
One product I did dig up, and which inspired my project for this investigation, promised to turn AppleWorks into HyperCard . When I learned of this program's existence, I had the reaction of a dog being asked, "Do you wanna go outside?"
StoryWorks , written by Robert C. Moore and published by Teachers' Idea and Information Exchange, promises we can "create incredibly powerful 'hypertext' applications (stacks) (and) create 'knowledge base' stacks." That is a tall promise for an 8-bit machine, and it is one the provided example files don't quite live up to. What it can do is create "choose your own adventure" style stories.
It takes a bit of boilerplate to adhere to StoryWorks's programming rules, and I will need to keep track of which "segments" (like HyperCard "cards") link to which. This sounds like perfect tasks for the database and spreadsheet to help manage. Then, I'll do some writing and formatting in the word processor, and next thing you know the potential and promise of AppleWorks's "integration" will be realized.
Or so the theory goes.
Navigating AppleWorks's file management can feel a little pre-Cambrian, not yet evolved into the complex, multicellular operating system you're reading this very blog within. Hardware-wise, we have the disk and the RAM. Files must be "on the desktop" to be used, which suggests some kind of "load from disk into RAM" step is necessary. That is precisely the case.
First, we have to let AppleWorks know which disk is our "current disk," shown in the upper left corner of the "desktop" screen. It's akin to interfacing with a single folder in your file system, and the visual metaphor of tabbed folders is used to assist with navigating the Desktop.
Once we have files on the desktop, fast switching via
OpenApple + Q
lets us bounce rapidly between documents making edits, while
AppleWorks
preserves our changes. Ready to quit? Not so fast, buddy. Sure
AppleWorks
appears to have preserved your work, but did you
SAVE YOUR WORK
? Thus far, our changes are only committed to the Desktop and, as established when we loaded from disk into Desktop, the Desktop is not the disk. From the Desktop, we need to "Save Desktop files to disk" to really save onto disk.
Educating Neanderthals
"The floppy is like a file cabinet. When you want to use a file, you get a copy from the disk, just as you'd take a file folder out of a file cabinet."
-
AppleWorks
Tutorial, pg. 14
Earlier, I noted the "default folder" and here's where it will come into play. Saving work to disk will save everything to the default folder, regardless of its original position on disk. I wound up duplicating files into lots of fun, new locations on disk, rather than overwriting the originals, as I learned the program. But, at least my files are safe. They were saved carefully, after all.
Two core tenets of AppleWorks 's approach to integration are its clipboard and keyboard shortcuts. Both work hard to bridge the natural divide between applications. The clipboard lets us move data between applications, and the keyboard shortcuts let us move muscle memory between applications.
This mostly works, as when doing cut/copy/paste, text insertion/deletion, finding information, printing, and other relatively generic tools. The shared keyboard shortcuts fall apart, somewhat, in their overly ambitious attempt to enforce uniformity across all apps, even when they don't make intuitive sense.
Let's look at the 'Zoom' command as a prime example. First, think about a command called 'Zoom' and envision what the result should be when invoked in a word processor context, then in a spreadsheet, and finally in a database. What did you picture in your mind's eye? What does it mean to 'Zoom'?
OpenApple - Z
will "zoom in" and "zoom out" between the record list and an individual record.
Contrast that with
OpenApple - F
to 'Find' text, which does exactly the same thing in all apps.
Keyboard shortcuts aren't evenly distributed across modules, either. Printer options are available in the word processor and spreadsheet, but
not
the database. "Find" works across all modules, but "Replace" doesn't work in the spreadsheet.
OpenApple - W
will split a window in a spreadsheet, ala
VisiCalc's
split window function. However, the word processor does not accept this command, though it would be a very reasonable expectation for that to work, as we saw in
PaperClip
on the Atari 8-bits.
I certainly recognize that each program will necessarily have unique needs. However, I'm not convinced the mental fortitude required to remember how one command differs from application to application is any easier or harder than just learning a new keyboard command specific to each application.
At this point, I've written thousands of words in AppleWorks and I can say confidently: the word processor is good. I have throttled the emulator to run at 1x speed, so I'm not completely head-in-the-sand about the reality of using the program on period equipment. Whatever horsepower I throw at it, it remains performant and a joy to type in.
Of course, we have the usual list of modern gotchas, like the lack of international character input, and the printer-centric formatting options (which do not survive ASCII export). But the basic act of getting words on screen is smooth, along with minimal chrome which keeps us informed of the state of the writing environment.
The chrome serves the document, ensuring the writer is never lost. The upper two lines show the ruler, what file we're working on, what mode we're in, and what hitting
ESC
will do right now. In the above case, here in the Printer Options screen,
ESC
will return me to the REVIEW/ADD/CHANGE screen. This clear explanation of what navigation buttons will do was something I appreciated about
Bank Street Writer
, and I'm happy to see a mature version of it in
AppleWorks
.
The rule shows our tab stops and their respective styles, but doesn't show an indicator for the right margin. Editing tab stops is as easy as editing a line of text, thanks to fixed-width type. Just type a character for the tab styling you want at the place you want it, including centering and decimal alignment.
The bottom displays a running line count and where the cursor currently sits. The bottom left has what seems like pretty useless information to me, "Type entry or use Apple commands." as a gentle reminder of how to use the program, I guess? The bottom right shows how to reach Help. That gives the UI four lines, with 20 devoted to writing. It is enough, though I do wish for a live word counter.
AppleWorks 3 brings integrated spell check, with reviews of the time saying it is better than any of the third-party spell-checkers that preceded it. That's a strange note, because I find its usage convoluted.
Invoked by
OpenApple - v
, to "verify" the document, the "Options" it presents are obtuse and unintuitive, but all its asking is if we want to replace words one at a time or in a list, and do we want a post-verification summary of everything changed? If so, how do we want the summary presented, printed or on screen?
The "Summary" is where we finally can see our word count, plus a list of all the weirdo words we used and how we corrected them individually, along with the occurrence count for each word. I'm not clear what I'm supposed to get out of knowing I used the word "turducken" 8 times. Maybe this is to help encourage the writer to mix things up? I refuse.
As is typical of word processors of the day, a vast amount of the program's features are related to printer formatting and control. I do not own a printer, and "printing" to an ASCII file strips away most of the fun stuff, like bold and superscript.
The spreadsheet portion is both amazing and boring. It's VisiCalc , with minor changes to meet the keyboard shortcuts (i.e. no slash menu command) and basic UI chrome of the suite. It does almost everything VisiCalc does, on a much larger worksheet and without quite as strict a memory barrier. If you have the RAM, even in this economy , you can eat VisiCalc's lunch.
It also has many of the same frustrations as VisiCalc , including row-based vs. column-based calculation order, which can require a full document double-calculation to synchronize early formulas with later cell values. We're also stuck with the archaic (even by this time) "one column width for all cells," rather than per-column width adjustments.
There are no graphing tools, and what options are available are so limited as to be effectively useless in a Lotus 1-2-3 world into which AppleWorks 3 was born; we'll need to rely on third party solutions to this problem. Copying complex formulas with relative cell references still requires manually selecting "relative" for every single reference in every single cell copied to a new position. Just a quick note to those cloning popular apps: you don't need to clone the terrible parts.
Still, VisiCalc once turned the Apple II into a must-purchase investment and birthed an entirely new genre of business software. Now, that watershed event is collapsed into one subset of bullet points in a longer feature list on the back of the product packaging.
The flat-file database module is uninspired, which is surprising to me because it is the module upon which this entire program was built. With a great word processor, and effectively a full clone of VisiCalc included, I expected similar depth from the database.
Setting up fields and records, searching, sorting, and filtering, are all simple enough to accomplish. Its adherence to AppleWorks's common keyboard commands makes it pretty trivial to search for records, edit text, delete records, and print. Yet there is so much it doesn't do, I can't help but feel disappointed.
To start, fields cannot be assigned value types, something even dBASE on CP/M could do years earlier. Everything is a string, without even the crudest form of data validation. AppleWorks 4 would gain the ability to at least set a field to be a number vs. text, though still no Booleans or field widths, for example.
When setting up a contacts database, a common use case is to have a "notes" field to remember things like family members, follow-up discussion topics, and the like. Thanks to a limit of about 70 characters per field, no single field in AppleWorks can hold that much information. I guess the solution is to set up half a dozen "memo" fields, just in case? That kind of workaround feels quite silly to me.
The remainder of the module is focused on generating "reports" and "layouts," the difference being "for printing" or "for screen." This has its place, perhaps to show a subset of data and focus on the important stuff, or to copy some data over to the word processor. Otherwise, there's just nothing to get excited about here. I can't even pretend to be excited for the purpose of writing a fun blog. Moving on!
The early days of productivity took some time to figure out how best to handle object selection and manipulation. On 8-bit systems especially, modality ruled the day. Today we've settled on the
object -> verb
paradigm, where we select an object to manipulate, then choose the manipulation.
Modality-driven software was the opposite. First, choose an action, then choose the object to affect, i.e.
verb -> object
. To move a paragraph, we first select
OpenApple + m
, to indicate our intention to "move." The system switches to a text selection mode, allowing us to highlight the text we wish to move. Finally, we position the cursor at the new location for the text, hit
RETURN
, and the text is moved. Its backwards, relatively speaking, but easy enough to adjust to.
Thanks to
AppleWork
s integration, we can copy/paste between any two Desktop documents. It's a little strange, because copy and paste are both considered "copy" operations, via
OpenApple + c
. We
copy to
the clipboard from application A, then we
copy from
the clipboard into application B, using menu prompts along the way to specify the direction of the copy.
Copying has its quirks, though. Copying between two documents of the same type behaves as expected. Across application types, copying into the word processor fares best, and most closely meets expectations. But we still encounter broken formatting, like how a line of text that spans multiple cells in the spreadsheet will break apart into discrete, tab-width sized chunks in the word processor.
What we actually want is to do is "print" to the clipboard, as backward as that sounds.
Depending on the module, different print options become available, to define what part of the data we want to print, and how to format it. Once done, printing to
The clipboard (for the word processor)
is an option. The end result is much cleaner data that is closer to WYSIWYG over the normal clipboard copy command. "Print to clipboard" is the only useful cross-application option, in my testing.
As the connoisseur of modern computer software that you are, I know you know that however good something is, it could always be a little better. Most software requires us to beg and plead for the developer to make the improvements that will ease our mortal burdens.
Some software embraces the notion of "extension," allowing itself to be a mere vessel for thoughts yet unthought, ideas yet unrealized. When you bought AppleWorks , it never occurred to you that you'd even want it to do outlining, until you saw ThinkTank and now you kinda wish you had something like that. AppleWorks is shockingly accommodating to our wishes.
One of the core features of Lissner's engine is a crazy-efficient memory management assembly routine. This gives, what by all rights should be "not enough computer," the superpower of fast app switching. Especially with a ProDisk (hard drive) attached, AppleWorks can become essentially a graphical shell for the Apple II.
A number of companies took advantage of this and created various add-on packages for AppleWorks . Beagle Bros, JEM Software, Pinpoint Publishing, and PBI Software collectively published over 100 extensions. They had a lot of competitive overlap, but provided add-ons like:
This was all well and good, but there was no official plug-in architecture for AppleWorks ; it was every developer for themselves. Installation routines were effectively "patches" to the base application, and there was no guarantee that patch A wouldn't step on patch B's toes during the installation process. It put some burden on the end-user to detangle things like installation order, or even compatibility with other add-ons of interest.
Beagle Bros wanted to solve that problem once and for all.
TimeOut was their solution to a simple, universal plug-in architecture for AppleWorks . Developers targeting TimeOut could rely on it to do the down-and-dirty interfacing with AppleWorks's memory management routines, without having to worry about patching or memory collisions. For the end-user, adding new TimeOut modules to their system was as simple as copying a single file to the AppleWorks directory.
This worked like a champ, and eventually led to Beagle Bros being the contractor for AppleWorks 3 . As the main app developer, they could steer its growth to align with their own vision, and so TimeOut became part of the foundation of AppleWorks proper. This then made it super simple to produce updates of significant value to the end-user, because those upgrades already existed in the form of TimeOut extensions.
And so, AppleWorks 3 received its first built-in spell checker as a result. As time went on, the outliner, expanded Desktop, better clipboards, and the like all became built-in features to AppleWorks .
The big one, from my perspective, is TimeOut UltraMacros. I have it installed as an extension here in AppleWorks 3 , and in AppleWorks 5 it became a built-in feature.
With it, we get a kind of mini-programming language, which includes if/then/else statements, variables, loops, value comparisons, and more than enough to fill out a 110-page manual. Macros work across all AppleWorks apps, and since the language is identical across apps, if you know how to script one, you know how to script all of them.
Still a few more years until the 6502 reaches its maximum potential.
With the powerful combination of AppleWorks + UltraMacros + StoryWorks , we have a veritable turducken of creative utility. In later writeups espousing the benefits of the AppleWorks plug-in system, it was suggested that future application authors would be writing exclusively with AppleWorks as the hosting application in mind. I can see the appeal.
Unfortunately, StoryWorks didn't get that memo, so the workflow between AppleWorks and StoryWorks isn't the seamless experience I would have liked. They are two completely separate programs; quit one to enter the other for each phase of the writing/debugging development cycle. Let's all take a moment to re-appreciate our multitasking operating systems.
Now, I've never written a Choose Your Own Adventure (CYOA) before. In browsing the books of my youth, I can see how
AppleWorks's
integration could be useful in designing such a thing. I can keep multiple documents open simultaneously, and jump quickly between them with
OpenApple + Q
. Here's my plan.
With the spreadsheet, I'll keep a list of page branches. This will sketch out the core content on each page, with page numbers showing where each choice leads.
In the database, I'll set up a template for the "programming" information for each page, based on the spreadsheet sketch. The database's 70-character limit on fields means I can't type the entire story into the database, but I can rough out a plot point or two.
Then, in the word processor, I'll merge the database into a template to save me from tedious, manual boilerplate formatting. Once merged, if I've done my job right, that should build a skeletal game I can dry run through StoryWorks .
"It's so simple, I can't believe people struggle to write games," the author scoffed.
OK, look, I was out of pocket and I take it all back. In the previous section I was a naive child, unaware of his own ignorance. That was "Then Me" and he was a fool. "Now Me" is wiser, less prone to shooting his mouth off about things he doesn't understand. Making games is hard, I can admit that now.
My biggest stumbling block is conceptually very simple: I don't know how to write a CYOA. What seemed like a clear plan in my mind turns out to not be that in practice. I thought, "Start with a rough sketch, start filling in details, repeat until done." would work.
My biggest stumbling block is conceptually very simple: I don't know how to write a CYOA.
So, fine, all this really means is that I'm not a game designer, a fact I actually already knew. Looking through websites about how to create such games, everyone seems to have a different take. Spreadsheets as an organizational tool comes up a lot, as does sketching out decision trees, for which the "Paint" add-on might work, but I have a better plan.
For my first attempt, I simply don't have the wherewithal to create something this intricate. My goal is to push the software, not my brain, so I'm going to be a thief and steal someone else's idea. Or rather, I'm going to steal someone else's structure.
The original Choose Your Own Adventure series, the "fourth best selling children's series of all time," was established by Edward Packard in 1976. Relaunched by Chooseco in 2010 , the classics have been reprinted in handy box sets, and even new adventures have been published. If you've ever wished for a Cthulhu-themed CYOA, aimed at readers 9 - 12, your very specific prayer has been answered.
The original CYOA books are also available on the Internet Archive's "Open Library" program. My intention was to get "inspired by" those, but instead I'm going to swipe the basic decision tree from one, and write my own story. There is a spy adventure called "The Deadly Shadow," by Richard Brightfield, that I will lean on for the heavy lifting of providing the structure for my story about low-rent super agent, James Epoxy.
Ah, let's just go ahead and call it a parody at this point. There's no point in kidding myself that I'm about to do anything original.
In working through my failed attempts, I encountered some frustrating limitations of AppleWorks integration.
In AppleWorks , we have an option to set numbered "markers" throughout a word processing document. These are invisible tags that can be jumped to, by number, for quick navigation to known locations. StoryWorks relies on these to delineate story "segments."
Just as AppleWorks can jump to markers, so too does StoryWorks use the same principle to jump to segments. The difference is that StoryWorks will only display text up to the next segment marker, effectively splitting the text into something akin to HyperCard "cards." And so, like HyperCard , StoryWorks also calls a collection of cards a "stack."
In setting up my word processing template, I added segment markers where appropriate. Merging in the test data went smoothly, the result of which was a new document written to disk as ASCII. Maybe you already smell the trouble? ASCII conversion stripped out all non-printable tags and markers, rendering it StoryWorks- incompatible. That means adding StoryWorks markers must be done post data merge.
Luckily, I have UltraMacros installed, so complex, repetitive, menu-driven marker creation is a simple self-defined keystroke away. In fact, because macros are just "a list of keyboard commands performed in order" I should be able to semi-automate find-and-replace placeholder text of my own design with a real StoryWorks marker.
If I run that a few times I can prep my document for StoryWorks ingestion lickety-split. Does anyone still use that term? Where did that phrase come from, anyway?
One thing I've quickly learned is that debugging is tricky. StoryWorks will parse the AppleWorks document and list its errors, but in practice I found that an early error can cascade, generating pages of errors. All we can really do is fix the first one and test again, hoping the later ones will disappear.
So the proper strategy is to test early, test often. Before there is anything resembling a cohesive narrative, we need to make sure the skeleton of the project, as merged from the database, compiles and works. For this purpose, it actually does help to have even truncated page descriptions populated from the database, just to get a yes/no understanding if navigation is working.
First, here's the template I've come up with.
^ [...]
and
^ <...>
placeholders in the template align with field names in the database. During a merge, possible field names are presented in a list for insertion, making it impossible to accidentally mistype one.
[]
means "don't insert a blank line if this database entry is blank."
<>
will always insert
something
, even if there is no entry in a record.
Here's what a representative record looks like. You can see the field names are referenced in the template, above.
R,r>84
is StoryWorks "code" for "When
R
or
r
are pressed, jump to segment 84."
As I mentioned earlier, the merge gives us an ASCII text file. That file, before adding true markers via macros, looks like this; note the
*MARKER*
text I use as a placeholder for where the macro should place a real page marker.
I see that the mail merge did not follow through on the promise of skipping blank entries. I also see that, despite showing me centered text on screen in my template, that was stripped from the document as well. I have quite a bit of cleanup to do to tighten up these layouts. This is becoming work.
Next, I'll run the macro I created to automate marker insertion. Macros are action-for-action transcripts of the keyboard actions required to accomplish a task, including cursor repositioning or text cleanup to prepare for the next step. Simple macros can be built simply, because macros are defined in the language of the AppleWorks user .
I cannot praise this approach to macro scripting enough, and I will bring it up every opportunity I can. The more I encounter it, the more I honestly feel something fundamental has been taken from users over time. It's one thing to allow someone to record their actions blindly, with a slightly patronizing "There, there, don't worry your sweet little head about how this all works."
It's quite another thing to tell a user, "Hey, you know all those tools you've been learning? You can use those exact same skills in powerful new ways." It would be relatively simple then to teach someone how to wrap an existing macro in a loop or decision construct ( UltraMacros can do these things and more), building up a foundation of self-confidence.
It must surely be far more accessible than whatever Google is proposing. I mean, speaking of Cthulhu!
After running my macro to the end of the document (holding down the macro shortcut auto-repeated until finished), here's the final result. I have to
OpenApple + z
to "zoom" into the document and reveal the hidden formatting codes, but so far so good. Everything appears to be ready for
StoryWorks
.
Fingers crossed.
0:00
/ 0:19
Hot dang, got it on the first try! (as far as you know) Now that the basic structure seems to be in working order, the last thing to do is write an entire novel.
*dry cough *
What a delightful piece of software; a delicious turducken! It will make excellent sandwiches for tomorrow's lunch. Easy to learn, easy to use, and even relatively complex cross-application actions are achievable with a gentle learning curve. I found I only needed manuals and books for very rare, "Can AppleWorks even do this?" questions, and very rarely for, "I'm stumped."
Even today, after six years of the Macintosh's existence, there has yet to be an integrated program as well-rounded and intuitive as AppleWorks for the Apple II machines.
— Steve Wozniak (A+, January 1991)
Of the word processors I've covered to date it's easily my favorite. Typing is responsive and editing is intuitive. I like the advanced tools, like markers, though I think other tools could be made easier to use or expanded. Keyboard shortcuts became second-nature very rapidly. Tasks I usually dread, like mail merge, are trivially accomplished.
The database " is ." It exists. It's fine. Perhaps its simplicity is a virtue, to some degree? I'm not personally smitten, but I'm glad to have it, and it did solve a real problem for me with the CYOA construction.
I enjoyed the spreadsheet as much as I did VisiCalc , which is to say it's very good but Lotus 1-2-3 still wins the day. Still, it's a lot of bang for your buck, considering it's only 1/3 of this package. Being able to easily share data with the other modules elevates its usefulness over VisiCalc . Its convenience gives it a clear win. Plus, it didn't include "Copilot integration" long before Microsoft removed it from Excel .
Perhaps the biggest takeaway for me is the stark reminder of how much can be achieved with so little: so little CPU, so little RAM, so little hard disk space. One can't help but ponder, "If this is what 128KB can do, imagine applying the same discipline toward 1MB of RAM."
Writing for inCider , Oct. 1989, Senior Editor Paul Statt compared the development cycle of Lotus 1-2-3 Release 3 (a rather notorious, oft-delayed "update") with AppleWorks 3 . Both were initially created by lone developers, Jonathan Sachs and Rupert Lissner. Release 3 of Lotus took a team of 40 developers years to write and shipped on 14 floppy discs. AppleWorks 3 was written by three Beagle Bros developers and shipped on two double-sided discs. An update to Lotus likely meant also updating one's computer to a 286 with 1MB of RAM and a hard drive. AppleWorks 3 ran on the exact same 128KB machine the version 1 ran on.
"It's too bad that software-industry pundits won't look closely at AppleWorks 3.0: They'd see software pacing its hardware. AppleWorks 3.0 proves how good software can be if it runs on— what can I call it but old?— hardware. Lotus 1-2-3 shows what happens when software developers struggle to remain 'state of the art.'"
Paul Statt, inCider, Oct. 1989
Almost 40 years later, his complaint feels uncomfortably modern, don't you think?
In recent news we read of ways to whittle Microsoft's 1GB weather app "down to" 130MB RAM consumption. While on the other hand, we have something like Weatherbot , occupying 6MB RAM on Windows 11 and about 2MB on System 7.5.5 (yes, it runs natively on both and more!)
The call for an efficient use of system resources is clearly still championed by some, but I don't see much of a call for brutal efficiency. Using 1/100 the RAM of a Microsoft-made app is fantastic, don't get me wrong, but let's get far more ambitious. No more thinking in megabytes, think in kilobytes ; express ambition by orders of magnitude .
AppleWorks singlehandedly kept Apple IIs in production use for decades , well into the GUI era, well past their "prime." We deserve such longevity again. We need it.
There is a financial cudgel of planned obsolescence that beats us down for lunch money on a regular basis. We're told it's for progress, but that proposition holds no tether to reality when we can see and touch a 128KB rebuttal that proves the bully a liar.
0:00
/ 0:41
Don't worry, I wouldn't leave you hanging. The beginning of my James Epoxy CYOA; pause to read. The description of Dimitrius is straight from the source book; I did not set that up for my running gag!
Ways to improve the experience, notable deficiencies, workarounds, and notes about incorporating the software into modern workflows (if possible).
.po
file extension, and
CiderPress2
can manage these, including copying file structures between disk images to cobble together your own custom disk from others.
As with many tools of this type and era, the program's emphasis on "printing" limits our formatting options pretty drastically. We can only embed non-printing printer codes into a document, and that requires setting toggles for (say) bold to start and stop at specific points in the page. It's anachronistic in all of the non-nostalgic, annoying ways.
/
slash menu; it could have been a nice alternate UI for those transitioning to
AppleWorks
. Editing cells, such as to change a cell from a label to a value, always tripped me up. Considering this is version 3, well after
Lotus 1-2-3
hit the scene, I do wish for better control over column widths and some concession to graph making. 3D spreadsheets, linking a cell in one sheet to an entirely different file, would be useful; this feature debuted in
AppleWorks 5
.
Late in my review cycle I came across a project keeping ProDOS alive on real Apple II hardware. The last official version of Apple ProDOS was 2.0.3 in 1993, but this project is at 2.4.3 with 2.5 on the way. John Brooks has been maintaining this for years now. It includes a kind of app fast launcher called Bitsy Bye which might smooth the process of switching between apps (like between AppleWorks and StoryWorks , in my case), if for no other reason than it appears to eliminate keystrokes and simplify file navigation.
1984 rocked.
Some thoughts on Dario’s post: 1. Dario does not actually address Gavin Baker’s account of what he said – something he could easily deny if it were inaccurate. 2. Dario claims his critics live in a “bubble” where all regulation equals regulatory capture. He calls this an overly simplified view and notes that “Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people.” This argument is a straw man. Of course treating all regulation as capture would be overly simplified – but almost no one holds that view. I have repeatedly argued for strong antitrust enforcement to keep industries competitive, especially Big Tech. If Anthropic continues toward monopoly or duopoly status, I would be among the first to demand those rules apply. 3. Regulatory capture is not vague or in the eye of the beholder. Nobel laureate George Stigler defined it as regulation acquired by an industry and designed and operated primarily for its benefit. Stigler challenged the traditional view that government regulation arises from a benevolent state protecting the public from market failures. Rather, industry groups have concentrated stakes and pour resources into influencing regulators, whereas the public’s stake is diffuse and unorganized. The revolving door between companies and the agencies that regulate them compounds the problem. Anthropic understands these dynamics: it has hired multiple senior Biden AI-policy officials and built a substantial government-affairs operation plus a network of aligned organizations to push its preferred frameworks at state and federal levels. 4. Dario has consistently pushed for a new federal agency to review and approve frontier models prior to release – a proposal framed variously as an “FDA for AI,” an “FAA for AI,” and most recently a “FINRA for AI.” I call it a “DMV for AI” because a review process modeled on the FAA or FDA (which takes years) or FINRA (which issues rules for a staid industry widely seen as protecting incumbents) will create long queues as AI models wait for testing and approval. This process will only become more labyrinthine as rules accumulate to prevent theoretical harms. This would handicap the U.S. relative to China, which will not adopt the same constraints. It would also undermine Anthropic’s own business model, whose pricing power depends on remaining ahead of open models. Whatever Dario states today, it is difficult to believe the company would simply accept outcomes that erase that advantage. 5. Anthropic is on track to become one of the most valuable companies in history, with the resources to navigate any approval process and shape the rules while competitors wait. Dario wants open models under heavier scrutiny – he has called them dangerous in Senate testimony, criticized them for not being centrally monitored or withdrawn, and linked them to IP theft. He says he has never sought a ban, but he could achieve a similar result by insisting that identical rules apply to both open and closed models. The U.S. risks becoming an island of costly closed models while the rest of the world races ahead with broader choice. 6. Dario acknowledges that AI is structurally centralizing but attributes this mainly to chips and scaling laws. Access to compute matters, but the deeper risk is who decides which capabilities are available to whom. His preferred pre-deployment testing and FAA/FINRA-style oversight would place that gatekeeping power in a federal bureaucracy working hand-in-glove with a small number of frontier labs – reinforcing centralization rather than countering it. 7. The second part of Dario’s post assumes we have amnesia about Anthropic’s well-orchestrated campaigns hyping AI fears. His May 2025 claim that AI would wipe out 50 percent of entry-level knowledge jobs within five years still lacks supporting evidence fifteen months later. Similarly Anthropic breathlessly promoted its heavily contrived “blackmail” study on 60 Minutes. Yet Dario blames public negativity on a long-standing loss of trust in institutions rather than his own messaging. 8. These narratives have done more than anything to shape public fear. People are left asking the same question Mark Zuckerberg posed: why race to build a future you describe in such negative terms? Thomas Sowell’s "The Vision of the Anointed" captures the mindset – elite intellectuals convinced that only they are enlightened enough to control the outcome. As Zuckerberg notes, concentrating power in the hands of an enlightened few has rarely produced the promised results; the practitioners turn out to be less enlightened in practice than in self-conception. 9. Gavin Baker summarized the disagreement cleanly on our pod: Dario believes frontier AI is too powerful to distribute; we believe it is too powerful to centralize. Dario appears to believe, sincerely, that safety and progress are best served by centralizing authority in a marriage of corporate and state power. The weight of human history gives us reason to fear that outcome.
1/2 Thanks Gavin for an especially thoughtful exchange. I don't usually spend much time on social media but I wanted to engage here because it really brings out the heart of an important conversation. First, on regulation, I think that “either concentrate it in the hands of a
Microsoft has reminded IT administrators that Windows Server 2022 is rapidly approaching its mainstream end date of October 2026, when it will switch to extended support.
The company announced Windows Server 2022 in March 2021, and it became generally available in September 2021 as the Long-Term Servicing Channel (LTSC) release, with 10 years of support.
"On October 13, 2026, Windows Server 2022 will reach end of mainstream support. The October 2026 security update will be the last mainstream support update available for this version," Microsoft said in a message center update on Friday.
"After this date, Windows Server 2022 will transition to extended support, which includes security updates at no additional cost, and will continue to receive monthly security updates through October 14, 2031."
It also advised admins to upgrade systems to Windows Server 2025 , which became generally available in November 2024 after first rolling out to Windows Insiders in January 2024 .
Windows Server 2025 will reach the end of support on November 13, 2029, with five years of extended support until November 14, 2034. Customers who want to test it before deployment can use the free Windows Server 2025 180-day trial available through the Microsoft Evaluation Center .
"Windows Server 2025 is now the latest Long-Term Servicing Channel (LTSC) release for Windows Server. To help keep your environment protected and supported, plan to upgrade to Windows Server 2025 for full mainstream support," Microsoft added. "Evaluate upgrade options and begin deployment testing early to help ensure a smooth transition."
Last month, Microsoft also announced that it has extended Windows Server 2022 hotpatching until October 2027, one year after its mainstream end date of October 2026, for systems running the Datacenter: Azure Edition, and quietly extended the free Windows 10 Extended Security Updates (ESU) program for consumers by an additional year.
More recently, it also reminded users that Windows 11 23H2 Enterprise and Education editions will reach end of updates on November 10, 2026, and recommended upgrading to Windows 11 25H2 (the company's latest operating system for endpoints).
You can find more information about Windows servicing dates on the Windows Lifecycle FAQ page or using the Lifecycle Policy search tool . Microsoft also maintains a list of all products that will reach the end of support or will be retired this year.
Overall prevention scores can hide what happens after initial access. Once attackers are using valid credentials, prevention drops sharply.
The Blue Report 2026 measures defenses technique by technique across 338 million simulations run in customer production environments.
This is a rush transcript. Copy may not be in its final form.
AMY GOODMAN : This is Democracy Now! , democracynow.org. I’m Amy Goodman, with Anjali Kamat.
ANJALI KAMAT : We turn now to the case of a leading human rights attorney in Britain who is facing criminal charges himself over his representation of a client in court. Rajiv Menon KC, King’s Counsel, is accused of criminal contempt over his closing speech at the trial of six Palestine Action activists for their direct action protest at a weapons factory owned by Elbit Systems, a supplier of arms to Israel, in Filton in August of 2024. Menon is being accused of misleading the jury and ignoring the judge’s directions in his remarks. He is believed to be the first defense attorney in Britain to be prosecuted over statements in a closing speech.
Mr. Justice [Nicklin]'s court order read in part, quote, “the respondent made statements which were capable of suggesting to the jury that the court was not impartial, in that he referred to the defendants being prevented from giving evidence about Elbit Systems, and later referred to Elbit Systems being ’protected' and 'wined and dined in the corridors of power.'”
AMY GOODMAN : Garden Court Chambers, where Menon has worked as a barrister for 30 years, said the charges brought against him were unprecedented and have, quote, “sent shockwaves through the legal profession,” unquote. The chair of the Criminal Bar Association has said the charges have left attorneys afraid of doing their jobs. Amnesty International said, quote, “The prosecution of Palestine Action lawyer, Rajiv Menon KC, is a serious threat to fair trial rights and the right to protest,” unquote.
For more, we’re joined by Rajiv Menon KC himself in his first television broadcast interview since the charges were brought. He’s joining us from London, where Democracy Now! just was.
Thanks so much for being with us. Why have you decided to speak out? In fact, you encourage your clients, Palestine Action protesters, not to speak directly to the press, is that right? And yet, now you’re the focus of the court.
RAJIV MENON : Firstly, thank you for inviting me.
I’ve decided to speak out because for the last few months a number of senior judges in this country have decided to make the most serious allegations against me in public, publishing those allegations on the Judicial Office website. And, of course, you’ve already highlighted the fact that I’m the first lawyer, as far as I’m aware, as far as my lawyers are aware, ever to be prosecuted for contempt of court in this — in British legal history. And I just feel that it’s important that I have a voice in this and that I at least address some of the matters in general terms that I’m accused of, as opposed to simply waiting, you know, for my opportunity in court months from now to say what I need to say.
ANJALI KAMAT : And, Rajiv, can you explain what the — why these contempt charges are being brought against you? What exactly did you say that was considered contempt of court?
RAJIV MENON : So, I made a closing speech in a Palestine Action protest case, as you said, on the 8th of January this year on behalf of my client Charlotte Head, who was one of those engaged in direct action against the Elbit Systems factory. And the judge subsequently made seven specific allegations of contempt against me. I’m — probably not wise for me to go into the full detail of that, but suffice to say, perhaps, that I categorically deny deliberately breaching any judicial order, and, as far as I’m aware, as far as I believe, didn’t inadvertently breach any order, either. I mean, contempt of court is a complicated jurisdiction, but, in essence, it involves a serious interference with the administration of justice. And I categorically deny that anything that I said in that closing speech amounted to a serious interference with the administration of justice. But that’s the allegation in a nutshell.
AMY GOODMAN : Rajiv Menon, since our systems are a little different, the legal systems not only in the United States and Britain, but around the world, if you can say in lay terms, in your closing argument, what is the argument you were making? I mean, we have a defense in the United States, where we have a description, for example, called jury nullification. They can say someone did something, but they don’t feel they should be found guilty. Talk about what the Palestine Action defendants were accused of, who you represented, and what your argument was.
RAJIV MENON : Yes. So, my client Charlotte Head was charged with aggravated burglary, violent disorder and criminal damage. And she had statutory defenses to aggravated burglary and violent disorder. The problem was, in relation to criminal damage, there was no dispute that she had participated with others in damaging property belonging to the Israeli arms manufacturer Elbit Systems. And the trial judge had withdrawn her only available defense of lawful excuse to that charge. And so, on the face of it, she had no defense to the charge. But, of course, she’s innocent until proven guilty. And ultimately, the facts are for the jury, not for the judge. The jury are the sole judges of the facts.
And so, in my speech, I told the jury about a very famous case from 1670 — excuse me — of William Penn and William Mead, who had been charged with unlawful preaching on the streets of London. And in that case, the trial judge had directed the jury to convict, and the jury had refused to convict. And the jury was subsequently imprisoned and, when they continued to refuse to convict, were fined. And some of those jurors refused to pay the fine. They were locked up. And that case, that came to be known as Bushel’s Case , is one of the most famous cases in British legal history. And I told the jury about that case, as hundreds, if not thousands, of other lawyers have done before me. And I told the jury about a plaque at the Old Bailey, probably the most famous courtroom in the world, where that case is celebrated — again, something that’s been done by countless lawyers before me.
And what I’ve been accused of is that by doing that, I was informing the jury about their right to acquit a defendant according to their conscience and was inviting them to do so. And that is an allegation I categorically deny. What you call jury nullification in the United States is called jury equity here. But I categorically deny — I need to make this absolutely clear — that I either informed the jury of the existence of that principle, that fundamental principle, or invited them to invoke it. What I did do was tell them about that famous case and about the plaque that celebrates the case. And that’s at the very heart of the allegation that I face.
ANJALI KAMAT : Rajiv, I want to turn to a clip of your client, Charlotte Head, speaking to AJ+ earlier this year.
CHARLOTTE HEAD : Palestine will never be forgotten. And for me, it’s something that I will never stop supporting and fighting for and wanting to scream from the rooftops. … We wanted to stop the manufacturing of weapons on British soil that were being used by Israel in the genocide in Gaza. I think we had asked nicely. We had asked so many times, in every different way, and the government wasn’t listening. … Having been in prison, which is obviously, you know, not even comparable to what people are going through in Gaza, but experiencing a smidge of that isolation, having people tell you that you’re not alone and that people are there and that you’re never going to be forgotten, I think that’s probably the most poignant thing I could say.
ANJALI KAMAT : Rajiv Menon, tell us about your client, Charlotte Head, and what exactly she was charged with. And she’s back in prison now?
RAJIV MENON : She is. Well, she’s an extraordinary woman — I should say that from the outset — who has dedicated most of her adult life to helping those less fortunate than her. She spent three years working in the refugee camps in Calais, doing a number of different jobs at the time to assist refugees and asylum seekers. At the time of her involvement in the action against the factory in Filton, she was working with women suffering domestic — and children, suffering domestic violence and abuse in London. So, she’s someone who’s dedicated, as I said, most of her adult life to helping others, and was driven to participate in this action by the genocide that was being live-streamed onto our televisions and our mobile phones in this country.
As I said, she was charged with aggravated burglary, violent disorder and criminal damage. She was acquitted by the jury at the first trial of aggravated burglary, which was the most serious charge that she faced, a charge that can technically attract a sentence of life imprisonment. The prosecution subsequently dropped the charge of violent disorder against her. The jury at the first trial were hung on that charge.
As far as criminal damage is concerned, she was — the jury couldn’t decide at the first trial, but she had a retrial later in April and May, and at that retrial, she was convicted of criminal damage and was given a custodial sentence of five years’ imprisonment. She’s currently serving that sentence.
And in addition to that, the trial judge found that her offending, quite extraordinarily — the first time this has ever happened in a protest case in this country — that her offending had a terrorist connection. And as a result, she’s subject now to a draconian regime within the prison system, which has, in real terms, lengthened the time that she will have to spend in custody. And that is a matter that’s currently subject to appeal.
ANJALI KAMAT : And the judge in this case, in the first trial and the second trial, put several restrictions on what was allowed as a defense. Is that right? Can you explain that?
RAJIV MENON : Yes. So, because the trial judge had withdrawn their justifications from them — as I mentioned earlier, Charlotte’s defense of lawful excuse was withdrawn from the jury; other justification defenses were withdrawn — the judge restricted what the defendants could say about their underlying motivations, and specifically about what they could say about Elbit Systems. And so, on a number of occasions during the first trial, defendants were stopped from telling the jury what they had learned about Elbit Systems prior to their involvement in the action. But they still managed to say a great deal. And to be fair, the judge never directed the jury at the first trial that the jury should disregard what they said.
At the retrial, when they were solely facing the charge of criminal damage, further restrictions were placed on them, and they were not allowed to say anything to the jury at the retrial about the reasons or underlying motivations that they had for either joining Palestine Action or participating in the action against the Filton factory or their specific views about Elbit Systems. So, that further restriction was placed on them at the retrial, that was not there at the first trial, where they were facing other charges, as well. I mean, it’s quite complicated, this, but I hope that explains it in a nutshell.
AMY GOODMAN : Before we end, we want to talk about that larger issue of Palestine Action being considered a terrorist organization in London. I was there last week, and I asked the British MP Jeremy Corbyn about Prime Minister Andy Burnham’s apology for the Labour Party’s initial stance on Gaza, and part of his response was about that crackdown on Palestine Action. This is what MP Corbyn said.
JEREMY CORBYN : If he is serious about a complete change in policy, if he’s serious — it’s a big “if” — then why are we criminalizing people who take part in protest in Britain? A number have already been convicted and face long stretches in prison. These are nonviolent direct action protests. And there are also probably 2,500 to 3,000 people who have been arrested for holding a placard, in contravention of the Terrorism Act 2000, which is basically labeling anyone that holds a placard up saying “I support Palestine Action,” calling them a terrorist. This includes people in their eighties and even nineties, retired Anglican clergy people and many others. It’s an absurd situation.
AMY GOODMAN : So, that’s British MP Jeremy Corbyn when we were in London last week. As we wrap up, Rajiv Menon, if you can talk about how you’re fighting the charges against you, the contempt of court charges, and also the chilling effect this has on how far lawyers will go to represent cases like these?
RAJIV MENON : Absolutely. Well, let me deal with the latter first. I mean, there is no question, as the chair of the Criminal Bar Association and the chair of the Bar Council in this country have both publicly said, that the prosecution being brought against me for contempt of court is having a chilling effect on criminal defense lawyers and the worries that they undoubtedly have these days, particularly in protest cases, about what they can and cannot say. And that’s obviously because of the unique, extraordinary and unprecedented action being taken against me. I mean, I think that’s irrefutable. I mean, there’s so much evidence about it. And literally hundreds of people have contacted me, lawyers and others, to talk about that chilling effect. And, I mean, that clearly is something that is of — you know, is extremely worrying. It’s been described by some as a descent into authoritarianism.
As far as the case against me is concerned, already the Court of Appeal in this country, in a judgment in May, has held that the trial judge in the Filton case and another senior member of the British judiciary, a lord justice of appeal, a member of the Court of Appeal, acted unlawfully against me in an excessive jurisdiction by allowing a direct referral to the High Court being made against me for contempt of court. So, we have already had that victory. The case then went back to the trial judge, who has now instituted contempt of court proceedings against me under a different procedure. We are now appealing against that, as well, on the basis that that also was unlawful. We’re awaiting a date for that appeal. We think it’ll probably be in October or November. Depending on the outcome of that appeal, of course, there are different possibilities. If I win that appeal, then this may go away. Alternatively, it may be referred to the attorney general to make a decision as to whether or not I should be prosecuted. If I lose my appeal, then I will stand trial for contempt of court at some stage either later this year or early next year, with the risk of a potential two-year prison sentence hanging over my head. So, this is a serious matter with serious constitutional implications, which is of tremendous worry and concern to members of the criminal bar and other members of the legal profession in this country.
AMY GOODMAN : Rajiv Menon, I want to thank you for being with us and agreeing to do this interview, against your own lawyer’s wishes, a leading British human rights and criminal defense lawyer facing an unprecedented contempt of court charges, and could be sentenced up to two years in prison for his closing speech for a Palestine Action defendant, speaking to us from London.
Coming up, The Nerd Reich: Silicon Valley Fascism and the War on Democracy . Stay with us.
[break]
AMY GOODMAN : “We Are Here” by Scott Donaldson and Richard Samuel Nolan.
The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
From: Con Kolivas <con@kolivas.org> To: linux-kernel <linux-kernel@vger.kernel.org> Subject: [ANNOUNCE] linux-7.2-ck1, MuQSS CPU scheduler for linux-7.2 Date: Mon, 17 Aug 2026 12:32:43 +1000 [thread overview] Message-ID: <CABqErrH=oQ3povVuSPhRON97v63=mB85jQmZjf443ofdYAuxxw@mail.gmail.com> (raw) Announcing the return of the first stable version of my out-of-tree patchset - not for mainline inclusion consideration. Tag: https://github.com/ckolivas/linux/releases/tag/v7.2-ck1 Tree: https://github.com/ckolivas/linux/tree/7.2-ck The -ck patchset aims to improve desktop/mobile device responsiveness, interactivity, and gaming, mostly by replacing the CPU scheduler en-bloc with my EEVDF, configurable runqueue sharing, MultiQueue Skiplist Scheduler. It's been 10 years since I originally abandoned the patchset for time reasons, but LLMs have made merging and development infinitely easier. Changes since the last publicly announced release are features I planned years ago and never implemented that are new: I/O aware CPU scheduling which accounts reads and writes to the calling task. Kthread work on behalf of a calling task is accounted back to that task. P/E core aware load balancing. Skiplist structure size minimisation & micro-optimisations. The mother of all resyncs to bring it up to 7.2. Numerous bugfixes. Note: Scheduler CGROUPs remain no-op stubs as they are largely unused in the target environments and would require massive amounts of code to support. Patchlist: Add -ck1 version. Make nohz_full not be picked up as a default config option and add recommendation to help. Set default Hz to 100 in combination with MuQSS and -ck patches. Make hrtimer granularity and minimum hrtimeout configurable in sysctl. Set default granularity to 100us and min timeout to 500us. Don't use hrtimer overlay when pm_freezing since some drivers still don't correctly use freezable timeouts. Replace all calls to schedule_timeout_uninterruptible to use schedule_msec_hrtimeout_uninterruptible. Replace all calls to schedule_timeout_interruptible to use schedule_msec_hrtimeout_interruptible. Convert msleep to use hrtimers when active. Convert all low value schedule_timeouts to their hrtimeout equivalents. Create highres timeout variants of schedule_timeout functions. Make preemptible kernel default. MultiQueue Skiplist Scheduler v0.31. The patches are kept modular for easy porting to each successive linux kernel version. Out of deference to LKML's signal to noise ratio, please just reply to me without CC'ing the mailing list for any discussion/issues. Enjoy! お楽しみください -ck
reply other threads:[~2026-08-17 2:32 UTC|newest] Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CABqErrH=oQ3povVuSPhRON97v63=mB85jQmZjf443ofdYAuxxw@mail.gmail.com' \ --to=con@kolivas.org \ --cc=linux-kernel@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).
yooper transplant - realtor - Nobel Peace Prize Nominee - follow for premium Twin Cities basement toilet content she/her
This is a rush transcript. Copy may not be in its final form.
ANJALI KAMAT : Operation Puppet Master. That’s the name of an expansive surveillance program launched by the federal government against activists, left-leaning political organizations, labor unions and other groups in Minnesota last winter as hundreds of thousands of residents joined protests demanding an end to the violent mass deportation campaign in the state by ICE and other federal agencies, known as Operation Metro Surge.
The spying program was launched by HSI , Homeland Security Investigations, four days after federal agents shot and killed Alex Pretti, a nurse and protester, and just weeks after agents shot another protester, Renee Good.
The spying program came to light in court documents filed in federal district court in Minnesota and first reported on by The New York Times . The documents were filed by defense attorney Kevin Riach in the federal case against 15 purported antifa members, Minnesotans accused of conspiracy to, quote, “impede or injure federal officers” and to, quote, “violently oppose immigration law enforcement.” Riach represents one of the accused, Isaac Sant.
AMY GOODMAN : In Kevin Riach’s motion, he reveals that undercover agents posing as protesters spied on community members at meetings in churches, parks, libraries, schools and union halls, and that HSI secretly obtained the financial records of national labor unions, including the SEIU and the Communication Workers of America, CWA , as well as national nonprofits like the Sunrise Movement. The financial records go back three years.
Democracy Now! spoke to Kieran Knutson, president of the CWA Local 7250, on Sunday to get his response.
KIERAN KNUTSON : We were outraged, but not surprised, to learn this news. You know, the federal government was conducting an illegal and murderous campaign of terror against immigrant and working-class people in the Twin Cities, so it’s not a surprise to us that they were using these methods to try and infiltrate and map the folks who were organizing opposition to their campaign. And, you know, it’s our understanding that they’ve surveilled our members, that they have infiltrated meetings that we co-sponsored, and possibly even entered our office once as undercover cops for the federal government as part of this operation. So, yeah, it just shows what kind of government we’re up against.
ANJALI KAMAT : In the motion filed with the court, attorney Kevin Riach writes, quote, “With no evidence, the government alleged to the grand jury that the conspiracy in this case extends far beyond the defendants to include the AFL - CIO , the Minneapolis Federation of Educators, the Minnesota Association of Professional Employees, Monarca, Veterans for Peace, and the Grease Pit bicycle repair shop, among others.” So far, none of the groups targeted by the HSI inquiry have faced charges.
The executive director of the Sunrise Movement, Aru Shiney-Ajay, said, quote, “While federal agents repeatedly broke the law, ordinary people exercised their First Amendment rights to protect their neighbors.”
The surveillance operation arose from a White House directive known as National Security Presidential Memo 7, NSPM -7, which identifies a potential domestic terrorist as someone expressing, quote, “anti-Christian, anti-capitalist, or anti-American views.”
AMY GOODMAN : For more, we’re joined by two guests. Isaac Sant is a healthcare worker in St. Paul, Minnesota, one of the 15 protesters charged by the federal government in June. It was his attorney, Kevin Riach, who filed this motion and released these documents on Thursday. And Emilia González Avalos is executive director of Unidos MN, a grassroots group and immigrant advocacy organization that was also spied on by the federal government.
We welcome you both to Democracy Now! I wanted to turn first to Isaac Sant in St. Paul. It’s your attorney who released these documents. It has been explosive around the country to see the federal government, after the killing of Alex Pretti and Renee Good, investigating who? The activists who were protesting these killings and the deadly immigrant crackdown. Now, your attorney advised you not to speak today, but you’ve chosen to. Why? Talk about what’s happened in the Twin Cities.
ISAAC SANT : Hi, Amy. Thanks for having me.
I believe this case is a blatant act of political repression against anybody who was brave enough to protest against ICE’s terroristic rampage through our streets in December and January. And I think the calculus here is a little different than in a normal criminal trial. I think these are politically motivated charges intended to quell dissent, and that my little organization is not really the target of these. I’m under indictment, and it’s obviously very stressful, but the whole community is really the target here. This is intended to have a chilling effect on protest and to intimidate people out of the street.
ANJALI KAMAT : Isaac, were you one of the groups that was under surveillance? And did you suspect anything during the time?
ISAAC SANT : I did suspect that we might have been under surveillance. I mean, I was at a mass meeting immediately after the brutal murder of Renee Good, and I thought there might be undercovers in the room, sure. But what I didn’t know was that —
AMY GOODMAN : What were you indicted for, Isaac?
ISAAC SANT : I was indicted for conspiracy to impede or injure a federal officer and also for interstate stalking.
ANJALI KAMAT : I want to bring in Emilia González Avalos into the conversation, executive director of Unidos MN. Emilia, can you talk about your organization? I’m looking at this chart of — that Homeland Security Investigations put out called “The Conspiracy,” and there’s a group called Monarca right at the top. And this is a project of Unidos, your group. Can you talk about your experience with federal surveillance?
EMILIA GONZÁLEZ AVALOS : Of course. Well, Monarca is, as you said, our constitutional observing program. It is an on-ramp for community organizing where we developed these massive forums for people to learn about how to legally and constitutionally observe what was happening in our streets with the deadly force being used upon our immigrant families. And that’s all we did. We were teaching people how to use their First Amendment; what does a warrant, an administrative warrant, looks like differently from a judicial warrant; what were the rights and opportunities that we have to protect the rights of immigrants in detention or while they are being detained; how do you locate a person that has just been detained? We also talked about Fourth Amendment rights.
And this program grew to almost 30,000 people before Metro Surge and reached to 50,000 people after Metro Surge, and it continues to grow. And so, we were shocked. We were sick to hear that we were also in this list of people that were organizing using their First Amendment rights and the right to petition to their government, but also not surprised. We have learned through history that this is one of the tactics they have used across the world to repress and scare and disrupt movements that are organizing with other people against racism, that are organizing for justice, and that are organizing for a multiracial, high-functioning democracy.
AMY GOODMAN : I want to go to what the U.S. attorney for Minnesota said in a news conference. He refused to answer repeated questions about whether any federal agents or officers were injured by the defendants.
DANIEL ROSEN : Whether or not they actually, at the end of the day, cause bodily harm is not the measure of whether or not they committed a serious federal crime. And I would dare say, you know, we just cannot have in this country all of all — people getting together, engaging in all of these violent acts, and then simply saying, “Well, you know, nobody got hurt, so how bad could it have been?”
AMY GOODMAN : That was the U.S. Attorney for Minnesota Daniel Rosen, who also suggested there could be more charges related to ongoing investigations into anti- ICE activism during so-called Operation Metro Surge.
DANIEL ROSEN : If you are actively conspiring to impede law enforcement, actively conspiring to commit the acts that today’s indictment alleges, you ought to go on the assumption that we’re watching you and that we’ll get you.
AMY GOODMAN : I mean, this is quite something. When there are accusations of violence, the U.S. attorney cannot name one example of violence that the defendants were involved with. Can you respond to this, Isaac Sant?
ISAAC SANT : On the advice of my attorney, I’m not going to talk about the details of any overt act I’m alleged to have committed in the indictment. But what I can tell you is that our activism, me and all of my co-defendants, was nonviolent throughout Metro Surge, and that, as I said, this is a naked act of political repression. This is not about what we did. This is about who we are and what we believe. This is about the suppression of our politics and, more broadly, of any opposition to ICE’s campaign of terror across the country.
And yeah, I didn’t hurt anyone. I’m not alleged to have hurt anyone. And there’s nothing in the indictment that I’m accused of that tens of thousands of people also didn’t participate in as acts of protest when ICE was rampaging through our streets.
ANJALI KAMAT : I want to bring Emilia back into this conversation. This indictment against the 15 defendants, you know, accuse them of everything from throwing ice chunks to setting up blockades. But the surveillance documents released last week now show the government was monitoring peaceful library meetings about deescalation tactics. What are your thoughts on the kind of evidence being gathered by the Trump administration and the financial records that have been collected of the SEIU , Sunrise Movement, three years of financial records?
EMILIA GONZÁLEZ AVALOS : Well, I am not an attorney, and I also have never experienced or witnessed a network of surveillance that tries to fish and hunt for the behaviors that they want to polarize, and then act using that, acting with the strength of the Department of Justice. It is also said that in this administration, a ham sandwich can get an indictment. And so, it is very dangerous, the times that we’re living, the discretion in which the rule of law is being used.
And I hold two things at the same time: Courts decide on individual conduct, and everybody has the right to due process. Everybody gets a defense. And a government that puts a bicycle collective and a teachers’ union and a network of regular people like Unidos on a conspiracy chart, in my opinion, loses the right to be believed about who the dangerous people are.
AMY GOODMAN : Emilia González Avalos, thank you so much for being with us, executive director of Unidos MN, and Isaac Sant, healthcare worker and activist in St. Paul, one of the 15 Minnesotans charged with conspiracy to impede federal officers during their immigration crackdown last winter. It was his attorney who released the documents.
Coming up, in an exclusive interview, the prominent British human rights attorney Rajiv Menon, speaking out on being charged with contempt of court for what he said during his closing argument in defense of his client, a protester with Palestine Action. We’ll go to London. Stay with us.
[break]
AMY GOODMAN : “I Keep Faith” by the British musician Billy Bragg, performing in our Democracy Now! studio.
The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
New apartment buildings, timeshares, and resorts are popping up all over Yucatán’s coast. Photo: Carlos Rosado van der Gracht / Yucatán Magazine
In a bold move that signals a major shift in Mexico’s approach to environmental protection, the Federal Attorney’s Office for Environmental Protection (Profepa) has launched an aggressive campaign against unauthorized real estate developments along the picturesque Yucatán coast.
“All works that aren’t authorized will have to be demolished,” declared Marina Boy Tamborrel, head of Profepa, during a recent meeting with Maya peoples in Ixil. This no-nonsense stance represents a dramatic departure from previous enforcement efforts that developers had learned to circumvent.
For years, wealthy developers have treated environmental fines as simply another line item in their budgets—a minor inconvenience easily absorbed by their profit margins. Tamborrel acknowledged this problem directly: “There are companies that allocate part of their budget to pay these fines and then continue operating; offenders prefer to pay the fine rather than request a permit.”
But those days appear to be ending. Profepa has already closed four real estate developments in Puerto Vallarta and promises similar action along all of Mexico’s coastal states. The agency’s new philosophy prioritizes environmental restoration over monetary penalties.
“The Attorney General’s new vision is to order the repair of the damage; it’s a priority,” Tamborrel explained, emphasizing that fines alone have proven insufficient to protect fragile coastal ecosystems.
The announcement comes as environmental officials prepare to assess damage in Sisal, where a staggering 23,000 hectares were recently devastated. The scale of this destruction highlights the urgent need for more stringent enforcement.
During the Ixil meeting, Tamborrel and Deputy Commissioner Wilberth Nahuat of Santa María Chi discussed the severe impacts suffered by local communities. They identified farms and irregular real estate developments as “the most critical issues” requiring immediate attention in Yucatan.
Of particular concern are projects operating without an Environmental Impact Statement (EIS), a required document that assesses potential ecological consequences before construction begins.
The crackdown faces significant challenges from developers determined to continue construction at any cost. Tamborrel cited one particularly troubling case involving a residential complex in Ixil: “We have open administrative files and we identified that they carried out new construction, so they did not respect the closure seals.”
This blatant disregard for legal orders demonstrates the entrenched resistance Profepa must overcome in its mission to protect Mexico’s coastal environment.
Recognizing that government agencies alone cannot monitor Mexico’s extensive coastline, officials are actively encouraging citizen participation in enforcement efforts. Residents can report violations via phone (800 776 33 72), email ( denuncias@profepa.gob.mx ), or through an online portal .
This community-based approach acknowledges the crucial role local populations play in identifying illegal activities that threaten both the environment and their way of life.
While it remains to be seen whether Profepa can successfully implement its ambitious enforcement agenda, the agency’s tough rhetoric and initial actions in Puerto Vallarta suggest a genuine commitment to reversing decades of unchecked coastal development .
For Mexico’s fragile coastal ecosystems and the communities that depend on them, this shift from symbolic fines to actual demolition of illegal structures could represent the dawn of a new era in environmental protection—one where developers can no longer treat ecological regulations as optional.
As Tamborrel promised the gathered residents of Ixil, “We will be very vigilant. It is a commitment and an obligation.”
Yucatán Magazine has the inside scoop on living here. Sign up to get our top headlines delivered to your inbox every week.
Last week, OpenAI announced the GPT-5.6 lineup, introducing the Sol, Terra, and Luna models. During the release stream , the team focused heavily on computer use , showing models capable of navigating and operating desktop applications. OpenAI highlighted UI agents and detailed 3D visualizations, but both depend on stronger visual understanding.
To measure their vision capabilities, we ran the models through our upcoming VLM benchmark, which we plan to release in the next few weeks. The benchmark covers common vision tasks, including detection, counting, OCR, and data extraction. In this post, we take a closer look at how GPT-5.6 performs across each of them.
Sol is clearly the best vision model OpenAI has released so far. The jump is especially visible in object detection and counting, where GPT-5.5 was far behind the strongest VLMs. Terra and Luna are not as strong as Sol, but both show meaningful progress over GPT-5.5.
Test Sol, Terra, and Luna in Roboflow Playground and compare their results with models such as Claude Fable 5 and Gemini 3.5 Flash across the same vision tasks.
Detection is where GPT-5.6 shows the clearest jump. GPT-5.5 scored 13.8 mAP@50 in our benchmark, while Sol reached 46.2. Terra and Luna followed closely at 44.7 and 43.3, moving object detection from a major weakness to a practical capability.
Document layout detection is one of the clearest strengths of GPT-5.6. Sol handled titles, paragraphs, tables, images, and signatures well. Many document workflows start with locating the relevant parts of a page before OCR or data extraction begins.
GPT-5.6 also performed well on dense scenes. The pills and eggs examples contain many similar objects packed closely together, a common weakness for VLM-based detection. Unlike traditional detectors, VLMs generate each class label and set of coordinates as text. As object count grows, the response becomes longer and the risk of missed objects, duplicates, or coordinate errors increases. Despite this, Sol detected most objects across both scenes.
For the best detection results, prompt GPT-5.6 models to return absolute XYXY coordinates in image pixels. This differs from Gemini 3.5 Flash, which performed best with YXYX coordinates normalized to a 0–1000 range. Using the wrong coordinate format reduced GPT-5.6 detection performance by around 15 mAP points in our benchmark.
In a few cases, GPT-5.6 Sol returned boxes in seemingly random parts of the image. Many had no overlap, or almost no overlap, with the ground truth. Instead of matching the visible objects, the boxes often formed unnatural layouts, such as straight rows or evenly spaced groups.
We shared those examples with OpenAI. Their team confirmed that Sol becomes less stable on images around 2,000 by 2,000 pixels or larger, especially at lower reasoning effort. Higher reasoning effort improves stability, but also increases token use, latency, and cost. Resizing or cropping large images before sending them to the OpenAI API is the most practical workaround.
Counting improved across the full GPT-5.6 lineup. Sol scored 73.0% in our benchmark, up from 64.9% for GPT-5.5, while Terra and Luna reached 67.6% and 66.2%. Luna, the cheapest model in the lineup, still outperformed the previous OpenAI baseline.
As part of the benchmark, we tested cases requiring more than spotting objects and returning a total. Sol counted heavily overlapping metal brackets, a difficult case for both traditional object detectors and VLMs. Sol also counted bullet holes only inside selected scoring zones, showing an understanding of both which objects to count and where the rule applied.
Blister packs proved much harder. In separate prompts, we asked Sol to count the empty slots and the pills still sealed inside the package. The repeated layout, reflections, and small visual differences between filled and empty slots made both tasks difficult.
The abnormal candy example exposed a different type of failure. Sol gave the wrong count, though it is unclear whether the model miscounted the candies or misunderstood the target category.
OCR performance stayed close to GPT-5.5. Sol achieved a 90.7% mean similarity score, only 0.5 points behind GPT-5.5 at 91.2%, while Terra and Luna reached 88.8% and 88.4%. The gap was larger in text extraction, where Sol scored 82.5% compared with 87.6% for GPT-5.5. Luna and Terra followed at 81.4% and 79.4%.
As part of the benchmark, we separated full transcription from targeted extraction. OCR asks the model to transcribe all visible text, while text extraction asks for a specific piece of information. Sol performed well on handwritten notes in both settings, producing a full transcription in one case and extracting a requested date in another.
Sol performed well on text embedded in complex visual scenes. It read a tire size sequence printed along the curved surface of a dirty, worn tire. In another example, it extracted the live score from a hockey broadcast and returned the answer in the requested format, testing both visual reading and instruction following.
Some simple-looking extraction tasks still failed. Sol could not read the expiration date printed on a blister pack. The text was small, vertical, low contrast, and affected by reflections, which may explain the error.
The vision gains come with higher token usage across the GPT-5.6 lineup. The difference matters less in small tests, but becomes more important at scale, where token volume directly increases processing costs.
Sol averaged close to 10 seconds per image in our benchmark. Terra reduced that to around 6 seconds, while Luna finished in slightly over 5 seconds. Luna offers the strongest latency-quality balance in the lineup, with speed close to Gemini 3.5 Flash while still outperforming GPT-5.5 on detection and counting.
In our benchmark, Sol cost roughly 2.5 cents per image, making it the second most expensive model after Claude Fable 5 . Terra reduced the average cost to about 1 cent per image, while Luna cost less than 0.5 cents.
At 0.8 cents per image, Gemini 3.5 Flash is much cheaper than Sol while still leading our detection and counting benchmarks. This makes it a strong option for data-intensive workloads where cost scales across large image batches. Roboflow Playground lets you test Sol, Terra, and Luna alongside Claude Fable 5, Gemini 3.5 Flash, and other VLMs on the same tasks.
With GPT-5.6, OpenAI is much closer to the leading VLMs than before. Detection moved from a weak point to a usable capability, and counting improved across the full model family.
There are still clear limits. Gemini 3.5 Flash remains a better practical choice for high-volume detection and counting in our benchmark, especially at its price.
GPT-5.6 shows OpenAI is now taking vision much more seriously. Sol still has flaws, especially around cost, latency, and some unstable detection cases, but the progress is hard to ignore. For agents, screen understanding, document workflows, and visual reasoning, this release makes OpenAI a much stronger option than before.
Watch Headlines
Aug 17, 2026
President Trump has reportedly threatened to “bomb the shit out of Oman” if it “gets in the way” of the U.S. blockade of Iranian ships in the Strait of Hormuz. It comes as Trump has downplayed concerns about conditions on the USS Abraham Lincoln, an aircraft carrier that has been deployed for more than 260 days, one of the longest deployments in U.S. military history. Two U.S. military newspapers — the Navy Times and Stars and Stripes — have reported multiple attempts by sailors to jump overboard amid deteriorating conditions aboard the Lincoln, including food rationing, plumbing backups and shortages of basic supplies including toothpaste and soap. This is President Trump telling reporters on Friday that the deployment hasn’t been long enough.
Reporter : “Family members of U.S. service members are concerned about the conditions onboard the USS Lincoln.”
President Donald Trump : “Well, no, that ship is moving. No, they’re not. That ship is moving right now, or very shortly, and it’s being replaced with another very similar ship.”
Reporter : “Has the deployment gone on too long?”
President Donald Trump : “No, no, no.”
Reporter : “Are you worried about the mental health concerns?”
President Donald Trump : “Not nearly long enough.”
This comes as President Trump said the U.S. will “substantially reduce” joint military exercises with South Korea, noting that Seoul had declined to join the U.S. war against Iran. He said his decision is based on his “very good relationship” with North Korea’s leader Kim Jong-un, adding that the country has been “unthreatening and respectful” as long as he’s been president.
Meanwhile, in Qatar, officials are denying Iranian accusations that Doha is secretly holding three Iranian airmen as prisoners of war. The dispute dates to the third day of the war, when Qatar said it shot down two Iranian warplanes it claimed were threatening its airspace.
On Sunday, Iran’s military announced a $30,000 bounty for killing or capturing U.S. soldiers, which would be doubled if it is carried out by a woman.
Amir Hatami : “Whoever, any dear combatant, kills an American or captures one and hands them over to any of the units of the Islamic Republic of Iran Army, supported by those dear individuals who wish to participate in financial jihad but under the guarantee of the Army, alongside the great rewards of jihad, we will provide a proper gift. This gift is equivalent to $30,000, approximately 5 billion tomans.”
Aug 17, 2026
In Lebanon, Israeli airstrikes killed at least 11 people, including children, on Saturday. It was one of the deadliest Israeli attacks on Lebanon since the so-called ceasefire with Hezbollah took effect in June. Israel has been striking Lebanon since early March after Hezbollah fired into Israel in retaliation for the U.S.-Israeli war on Iran. Israeli strikes have killed more than 4,300 people in Lebanon this year. According to the U.N., more than 1 million people have been forced to flee their homes.
Aug 17, 2026
President Trump’s son-in-law and envoy Jared Kushner has met with Israeli Prime Minister Benjamin Netanyahu after Israel rejected a 15-point plan for Gaza promoted by President Trump’s so-called Board of Peace. The proposal would see Hamas gradually disarm and turn over governance of the Gaza Strip to an international force in exchange for Israel’s withdrawal. Over the weekend, Kushner met in Cairo with Hamas’s political chief Khalil al-Hayya. Also joining the talks were Egyptian President Abdel Fattah el-Sisi, former U.K. Prime Minister Tony Blair and Qatari, Turkish and Egyptian mediators.
Aug 17, 2026
In the occupied West Bank, Israel’s military has announced plans to hand over responsibility for policing Israeli settlers to a civilian police force. Palestinian officials condemned the plan as a flagrant violation of international law, calling it another step toward Israeli annexation of the West Bank. This comes as a siege by Israeli settlers on the village of Qusra has entered its second week, with 15 Palestinians, including two children, cut off from food, water and electricity by armed Israeli settlers backed by soldiers. Human rights groups say it’s part of a concerted effort by Israelis to seize even more Palestinian land. This is a video filmed by Aysha Hassan, whose family has been confined to their home for over a week while Israeli soldiers occupy a neighboring building.
Aysha Hassan : “We are in our besieged house and have been under siege for the last seven days. The soldiers are stationed on the nearby house of my brother Yousif. They keep screaming and singing all day.”
Aug 17, 2026
In Russia, officials say at least nine people were killed over the weekend when Ukraine launched one of its largest drone attacks since the start of the war. Moscow’s Defense Ministry claims it shot down nearly 1,500 Ukrainian drones over 24 hours. A massive fire tore through a distribution warehouse operated by the Russian online retailer Wildberries just outside of central Moscow. Ukrainian officials say Russia struck 13 locations across Ukraine over the same period, killing seven people.
Aug 17, 2026
In Afghanistan, the Taliban have marked five years in power since U.S. forces withdrew from the country. Since retaking control in 2021, the Taliban have stripped away the rights of women and girls, with UNESCO reporting some 2.4 million girls are now barred from secondary and higher education. The United Nations warns more than half of women’s organizations still working in Afghanistan could cease operations within the next year due to funding problems. This is the permanent representative of Denmark to the United Nations.
Christina Markus Lassen : “The Taliban’s systematic repression of women and girls has become a defining feature of the current system. Afghanistan remains the only country in the world where girls are banned from secondary and higher education. The grave abuse of the right to education is also having far-reaching consequences for Afghanistan’s economic development and provision of essential public services.”
Aug 17, 2026
Hundreds of migrants in Spain’s North African enclave of Ceuta gathered for a protest Sunday demanding they be granted asylum. They’re among the thousands who remain in Ceuta after more than 70,000 people crossed into the enclave in recent weeks seeking protections. Dozens died, mostly from drowning, in their attempt to reach the Spanish territory amid an intensifying crackdown along the border with Morocco. Many of those who stayed in Ceuta are now forced to sleep on the beach or in the city’s outskirts. This is a migrant from Morocco who swam to Ceuta.
Ayoub Elarroud : “We have problems and bad situations in our country. There are so many talents in our country that have been lost. … We are truly suffering. We are sleeping next to the sea, with cold, with bad smell. We are suffering a lot. The police doesn’t let us go downtown. They always kick us out, telling us to get the hell out of there.”
Aug 17, 2026
In Indonesia, a 7.7 magnitude earthquake that struck the country Saturday was followed by nearly 1,000 aftershocks, killing more than 50 people. The largest earthquake hit just north of Flores island, followed by a second 6.4 magnitude quake in Indonesia’s western Sumatra island. An estimated 5,000 people have been evacuated as search and rescue teams race to find survivors. Many evacuees have been forced to sleep in the streets, afraid of more aftershocks. Meanwhile, tents have been set up amid the rubble of collapsed hospitals to serve as makeshift emergency rooms to treat injured patients. Landslides have also blocked roads in parts of Flores island, raising concerns of dwindling aid and resources as people go hungry.
Febronia Aselmia : “My husband in the morning took Rohini’s baby and went to the store, and he got us three loaves of bread to share. One loaf of bread — I felt so sorry for my nieces and nephews — had to be divided, little by little. She’s pregnant. I feel so sorry for her, so I only ate a small amount. Even a single banana would be divided among three people. We were desperately hungry.”
This is Indonesia’s deadliest quake since 2022, when an earthquake in West Java killed hundreds.
Aug 17, 2026
Image Credit: Virginia State University
In northern Michigan, a gunman killed five people on Friday, prompting a manhunt that ended when the shooter was found dead alongside one of his victims. Separately, police in Petersburg, Virginia, arrested a 19-year-old suspect on Saturday following an overnight shooting at Virginia State University that left five people injured — one of them critically — just hours after officials welcomed freshmen to campus. According to the Gun Violence Archive, the U.S. has seen at least 305 mass shootings since the start of the year.
Aug 17, 2026
Image Credit: Daniel Torok
President Trump’s former personal lawyer, Attorney General Todd Blanche, has declined to pledge that the Justice Department will operate independently of the White House. Blanche was asked about the topic by NBC’s “Meet the Press” host Kristen Welker.
Kristen Welker : “You, of course, used to be President Trump’s former personal defense attorney. Can you pledge that the Justice Department will always act independently of the White House?”
Attorney General Todd Blanche : “Well, what — that’s — there’s a big difference between saying we will be — we will always do our job and investigate any case and act independently of the White House. No, I’m not going to pledge that.”
Aug 17, 2026
Image Credit: Reuters
A federal judge has ruled that the state of Idaho must allow abortions when the pregnant person’s health and life are at risk. U.S. District Judge B. Lynn Winmill ruled Idaho state officials cannot prosecute physicians who perform abortions under these circumstances, saying Idaho’s near-total abortion ban violates the due process and equal protection clauses of the 14th Amendment. Physicians faced up to five years in prison if found in violation of the ban. In his ruling, Judge Winmill wrote, “It is about self-preservation and the limit of the state’s power to make a woman suffer for the sake of an unborn child. … A pregnant woman’s health is not a state resource to be allocated at the legislature’s whim.”
Aug 17, 2026
The Democratic National Committee has approved sweeping changes to the 2028 presidential primary calendar. Under the new schedule, South Carolina voters would cast the first ballots on January 22, followed by Nevada on February 1. Iowa and New Hampshire would no longer be the first states to hold nominating contests. The move could expand the influence of Black and Latino voters on narrowing the field of presidential contenders.
The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
MargJohnsonVA/Shutterstock
Once a juggernaut of the home computer, Commodore is now a niche boutique brand, run by a YouTuber. So how did we get from the immensely popular Commodore 64s through the 80s, a classic gaming experience with the Amiga, and now, a flip phone ? Well, to say it's simple would be lying.
Commodore was originally founded in 1976 by Jack Tramiel and Irving Gould. A year later it launched the PET and followed it up with the VIC-20, which was one of the first to hit a million units sold. In '82, it launched the Commodore 64, which became a powerhouse amongst home computer users. While the Commodore 64 was the United States' home computer of choice through parts of the 80s, the '85 jump in tech with the Amiga 1000 gained huge popularity throughout Europe. By 1996, the Amiga was now a niche, as the PC market had completely dominated the computing space, and home consoles were far cheaper, and easier to operate.
With the company bankrupt , this is where it begins to get a little complicated. Originally, a German company called ESCOM, backed by companies based in Asia, essentially acquired everything in 1995. ESCOM was reportedly attempting to revive the Commodore Amiga, partnering with one of the Asian companies, Tianjin Family. In 1996, ESCOM itself went under, selling to the Dell-alternative Gateway . The cow-themed PC maker got everything bar the "C= Commodore" trademarks and name, which eventually wound up being owned by a Netherlands-based company, Tulip Computers.
Michael Ochs Archives/Getty Images
During this time period, the Amiga operating system was still licensed out to the company Village Tronic Marketing GmbH. Gateway sued to ensure that sales stopped, but it was found that there was no documentation supporting Gateway's argument. Eventually the company was forced to stop selling over a payment dispute, rather than an elaborate court case. Between 1997 and 2005, three different companies branded with Amiga popped up under the Gateway watch.
South Dakota was Gateway's main Amiga branch, but was merged back into the company. It was intended to develop the next two versions of AmigaOS, 4 and 5, but never released anything. Gateway closed it in 1999, selling everything to Amiga Inc., based in Washington. This was more of a license holder than anything, and Delaware was a rebranded KMOS, Inc.. While brands and rights changed, Gateway still held the patents. In 2007, when it sold to Acer, the Taiwanese company took ownership.
Prior to where we are now, the Commodore brand's main regular news was legal disputes over the state of AmigaOS or resurrections. In 2001, a company called Hyperion was contracted to develop AmigaOS 4. In 2007, the company was sued over unpaid royalties by the Cloanto owned Amiga Corporation. Hyperion has only just settled the various legal issues as of writing, today, August 4, 2026. In 2010, Commodore USA was started, releasing hardware, like the 64x. These were poorly reviewed and never sold enough, ending its short run in 2013.
Prior to the latest iteration, Retro Games, produced the C64 Mini and C64 Maxi , as well as A500 and A1200 Mini , based on Amiga systems. These were licensed out by Amiga Corporation (Cloanto). Cloanto also produces Amiga Forever and C64 Forever, emulation packages. There was also AmigaOne, built on newer PowerPC processors, designed to work with Hyperion's AmigaOS 4.
As of today, Commodore is back once again. In 2025, a YouTuber called Christian Simpson started Commodore International. Bringing back individuals from the original company and focusing on bringing the brand back under "two pillars," nostalgia and the future, has already gained momentum.
Its first product, the C64 Ultimate, uses an FPGA processor to (boiling it down for simplicity) emulate the original hardware, rather than using software. This was well received, and the company has already gotten up to speed with legal litigation too, suing an Italian company over its name, which it claims was "improperly granted." It now plans to launch a phone, the Callback, which blocks social media use. Reactions to it online have been divisive, and a price drop was announced not long after its initial pricing.
Commodore and the subsequent Amiga branding is a tangled web, and this barely scratches some of the weirdness in the intervening years. If this were fiction, it'd be lambasted for how confusing it is.
For help please visit help.ft.com . We apologise for any inconvenience.
The following information can help our support team to resolve this issue.
Tech giants General Electric (GE) and Philips have also confirmed they're investigating claims that the Clop ransomware gang breached their systems and stole data.
While a GE spokesperson said the company is aware of the claim and is "working to assess the potential issue," a Philips spokesperson confirmed its systems were breached but said the incident has been contained and didn't affect customers.
"Philips has identified and contained an attempted cybersecurity compromise of a specific enterprise server related to internal data," Philips said in a statement shared with Reuters . "This has no impact on customer environments."
GE and Philips spokespersons have yet to reply after BleepingComputer also reached out to them for more details and to confirm the Clop ransomware gang's claims.
This comes after oil giant Shell also said on Friday that it is investigating a potential security incident after the Clop hacking group claimed it stole 89GB of data.
"We are aware of a potential incident," a Shell spokesperson told BleepingComputer when asked to confirm the gang's data theft claims. "We are working with our security teams and relevant experts to investigate.
While the three companies have yet to share more information, the Clop gang has listed them on its leak site as part of a batch of 43 new victims likely targeted in data theft attacks exploiting a critical improper input validation vulnerability (tracked as CVE-2026-12569 ) against Internet-exposed PTC Windchill and PTC FlexPLM instances .
PTC says the two enterprise software platforms are widely used by high-profile companies across the aerospace, defense, automotive, heavy machinery, retail, and medtech sectors. The company says more than 30,000 customers globally use its products, including over 1,500 brand and retail customers using FlexPLM.
In these attacks, Clop claims it stole a wide range of sensitive data from the companies' compromised systems, including backups, project plans, photos of facilities, drawings, diagrams, blueprints, and more, belonging to Shell, GE, and Philips.
PTC began releasing CVE-2026-12569 security patches on June 17 and urged customers to review environments for indicators of compromise (IOCs) in a private advisory , even though there was no confirmation of in-the-wild exploitation.
Since then, cybersecurity company ReliaQuest and t he Ransomware Information Sharing and Analysis Centre (Ransom-ISAC) have confirmed Clop's Windchill and FlexPLM attacks, in which the threat actors have been deploying JSP webshells to steal sensitive data from victims' compromised PLM platforms.
The U.S. Cybersecurity and Infrastructure Security Agency (CISA) also confirmed that the flaw is actively exploited in attacks after PTC warned of "heightened threat activity" on June 26, mandating federal agencies to secure their PTC Windchill and FlexPLM instances within three days after adding it to its catalog of known exploited vulnerabilities .
This vulnerability has also prompted emergency action from German authorities , with the Federal Office for Information Security (BSI) warning PTC customers in the middle of the night to patch systems as quickly as possible.
The Clop extortion gang has a long history of targeting enterprise platforms in data theft attacks, breaching Accellion FTA , GoAnywhere MFT , SolarWinds Serv-U FTP , Cleo , and MOVEit Transfer file-sharing servers in previous campaigns, with the latter affecting over 2,770 organizations worldwide .
Starting in early August 2025 , it also began exploiting an Oracle EBS zero-day flaw to steal sensitive files from many organizations. The list of victims includes many high-profile organizations worldwide, including The Washington Post , GlobalLogic , Harvard University , the University of Pennsylvania , Logitech , Estée Lauder , Korean Air , and American Airlines subsidiary Envoy Air .
The U.S. Department of State now offers a $10 million reward for any information linking the cybercrime gang's attacks to a foreign government.
Overall prevention scores can hide what happens after initial access. Once attackers are using valid credentials, prevention drops sharply.
The Blue Report 2026 measures defenses technique by technique across 338 million simulations run in customer production environments.
Criminals are hacking into public Wi-Fi devices—at hotels, conference centers, and so on—around the world and changing their DNS settings. The goal is to redirect users to fake login pages and steal their credentials.
Tags: credentials , DNS , hacking , phishing , theft
Subscribe to comments on this entry
Sidebar photo of Bruce Schneier by Joe MacInnis.
2024-02-04
I'm going to describe the various tricks I used to write fast compilers for my programming languages. By fast compilation, I mean compiling at least 500.000 lines of code per second (excluding blank lines and comments) on a single CPU core.
You may argue that compilation time is not important. After all, once released, who cares that a program took hours to build; as users, we only want it to work and to work fast. It's like complaining that the last Pixar movie took days for the final rendering.
However it can severely affect the development cycle and make developers angry. It's 2024 and I can see that the most common complaint for Rust is still its compilation time.
The speed also affects the design of the compiler: when my biggest program is less than 100K SLOC and I can compile 500K SLOC per second I don't really have to worry about separate compilation since a complete build takes less than 200ms. And this is good since separate compilation can be tricky with genericity.
If you're writing a compiler for an existing language, e.g. C++, you have no control here, you'll have to deal with an LL(k) grammar, a preprocessor and a terrible module system. Conversely, if you're writing a compiler for your own programming language, careful design choices can help a lot.
I've always used a context free grammar that can be easily parsed with a simple recursive descent parser. If a syntax is easy to parse by the computer it will be also easy to parse by a human.
A simple syntax will also make the development of independent tools easier (static analyzer, formating tools, refactoring, syntax highlighting, ...).
Less code to execute and less memory access usually means faster execution. While modern architectures don't make this principle strictly true, it is still a good rule to follow.
I avoid copying data as much as possible. Many languages use zero-terminated strings. I prefer to use a pair of ( start_pointer , size ) or ( start_pointer , end_pointer ) instead: it allows for instance to refer to any sub-string directly from the input buffer without having to do a copy.
The less memory I use, the more it will fit in cache.
A lot of work in the compiler is to check for errors but a program has usually no error or very few ones. Therefore the code must be optimized considering that errors are exceptionals.
A memory region is a contiguous block of memory where parts can be allocated but not de-allocated (the entire region must be de-allocated). The allocation consists just in advancing a pointer, and eventually creating a new region when the region is full. It makes allocations extremely fast. The de-allocation is also extremely fast since all objects of a region are freed at once.
This kind of memory management fits very well with a compiler: a single region can be used for a compilation unit. In practice I use 3 regions:
However when compiling functions, there are lot of temporary objects created, mainly dictionary of names for each lexical scope. To handle this I create pools of regions: instead of creating and destroying regions I pick one from a pool and put it back to the pool when finished.
Resizable arrays and open addressing hashtables are not suitable data structures for memory regions since they need a lot of de-allocations and re-allocations.
To store lists of elements, when possible I count the elements first and then I allocate a fixed size array, when it's not possible I just use a link-list.
To handle hash tables which are heavily used for names, I use separate chaining instead of open-addressing. It eliminates re-allocation of arrays but it requires to carefully choose the size of the hash: the global namespace will need a bigger hash table than the inner scope of a function.
CPUs are not designed to work with strings, they are designed to work with fixed size integers.
An important part of the compilation process is to find which entity is associated to an identifier. If the identifier is stored as a string, it will be slow.
The compiler does not need to know the content of an identifier except to present it to the user when an error occurs or to store it in a debug symbol table in the object file. It only needs to test equality between identifiers, so it can work internally with an integer assigned for each distinct identifier.
Since the lexer already needs to do a lookup in a dictionary for keywords, the additional cost to convert an identifier into a unique integer during lexical analysis is very small.
The following source code
func main(args)
if args.size < 2
will produce this flow of lexical units:
where:
main
,
args
and
size
.
I use a simple classic architecture for my compiler:
Lexical Analysis ---> Syntax Analysis ---> Build ---> Code Generation
where:
With a carefully designed grammar, the lexical analyser is just a loop with a giant switch statement processing input characters and the syntax analyser is a hand-coded recursive descent parser combined with a grammar of operators for infix expressions.
The recursive descent parser is made of functions like this:
e = parseExpression
i = parseInstruction
b = parseBlock
id = parseIdentifier
...
The natural way to handle errors is to return a special value when there is a syntax error. e.g. returning a null value, but it forces every caller to check the return value.
e = parseExpression
if e == nil
return nil
end
The trick is to report an error and return a dummy expression instead. e.g. returning a '0' expression. The syntax error function will display the first error and will drop the next ones since they can be completely unrelated to the input.
Not only the code is faster by eliminating many tests, it is also simpler: it is not cluttered with test at every line, it has the advantages of exceptions without the drawbacks (hidden exit points, cost of stack unwinding).
This trick comes with a cost: I stop at the first syntax error (contrary to the build step where I try to report as much errors as possible).
When importing a library, you usually end up with tons of constants, structures and functions that you're not using.
It's a good idea to analyse only what's needed.
The syntactic analysis has to read and parse the whole input in order to create the AST.
class Label: Widget
attr caption: String
attr color: Color
def init(caption: String)
...
// ... hundreds of lines of code ...
end
But for the build step it is just:
The content of the class and its parent can be completely ignored until needed. The compiler just needs to associate Label with a yet undefined entity, so it can find it when needed and ensure that no other entity will be defined with the same name.
It means that the compiler won't validate code until it is used. It can be annoying in some case, for instance when writing a library you want to be sure that everything compiles. A command line option can force the compilation of everything.
Also the design of the language can help here. By not having everything in the global namespace, you can considerably reduce the number of declarations that need to be defined. For instance GTK+ is a library with a C API but in my language I can expose many functions as methods in their respective Widget class:
Original C API
typedef struct _GtkLabel GtkLabel;
GtkWidget* gtk_label_new (const gchar *str);
void gtk_label_set_text (GtkLabel *label, const gchar *str);
const gchar* gtk_label_get_text (GtkLabel *label);
...
In Copper
class GtkLabel: GtkMisc
import func "gtk_label_new" new(String): Self
import def "gtk_label_set_text" set_text(String)
import def "gtk_label_get_text" get_text: String
...
end
If
GtkLabel
is unused, all nested entities won't even be declared.
With huge libraries where only few elements are used, it can save a lot of time.
In addition to a LLVM and C backends, I've developed my own x64 code generator.
LLVM is great; it is very easy to use, it generates highly optimized code and supports many platforms. But it is slow, I've observed a factor of 30 speed up by developing my own x64 backend.
My LLVM backend has 2000 lines of straightforward code, while my own x64 backend has more than 10.000 lines of complex code and it does not even support floating point numbers. It has been a huge effort to develop my own x64 generator but it was worth it. May be someone should develop a non-SSA alternative to LLVM to allow implementation of fast compilers.
My code generator is mainly made of:
An intermediary pseudo-code is generated, it makes some optimizations easier. It can be directly assembled to machine code, no need to translate it into assembly code to pass it to an assembler.
It is mostly like x64 assembly but with variables. After register allocation, variables will be replaced by registers or by a memory access. This pseudo-code is not generic like the IR of LLVM, it is optimized for x64: for instance the indexed memory accessed is used.
This part is necessary because of the genericity in my language that creates many functions with different data types but generates identical code. On one hand this duplicate finder costs a lot in performance as finding duplicates in a graph with potential recursion can be tricky; on the other hand it eliminates a lot of functions to assemble.
I use the linear scan algorithm (PDF) for register allocation. The original paper describes it as intended for JIT compilers but in my opinion it is also a perfect fit for fast compilers that want to generate reasonably fast code.
By assembling in two passes, it seems slower at first, but in the first pass I don't write anything: I don't need to use resizable growing buffers, I just have to count bytes to compute offsets. In the second pass, I know the exact size so I can pre-allocate a buffer with the right size and write without checking limits. The cost is that the write functions are virtuals. Without a real benchmark I can't tell wether a two-pass is faster than a one-pass but I found that the two-pass is not too bad.
Compiling my projects in 100ms is good enough so I don't need any more optimizations for now. Anyway, all the tricks described above are just basic techniques and common sense, there is still a lot of room for improvement and experiments:
I hope it will be helpful if you plan to write a compiler or if you want to make an existing one faster.
Andrew · 17 August 2026 · 3 min read
I have been reflecting on a large optimization I built for my game with the assistance of a frontier coding agent. This optimization took me several days to plan with the agent, about a week to comprehend the massive diff it produced (I review and approve every line), and another week to refactor and refine it until completion.
By the end, I understood everything that had been built—as if I had written it myself—but I was burned out. Untangling and comprehending large amounts of clever, high performance code and deciding if it makes sense within each stratum of abstraction is taxing on the mind.
If I had built it entirely unassisted, I suspect it would have taken me about a month. It would have been difficult, but I would have paid the comprehension costs as I developed it, in manageable chunks, and burned out much more slowly.
I asked Claude make me an interactive visualization for how I perceive these relationships of coding, comprehension, and burnout:
Drag each slider to see how different factors can relate burnout.
Model comparing the time and burnout cost of writing code by hand versus reviewing generated code, at varying levels of vetting rigor.
? How much of the generated code you actually understand before shipping. 0% is trusting the diff; 100% is knowing it as well as code you wrote yourself. 100%
? How close the generated design is to the one you would have written — your decomposition, your conventions. High alignment makes review cheap and rework rare. 40%
? Scope of the change, 1 to 10. Drives baseline hours superlinearly — a 10 is not ten times a 1, because the pieces interact. 6
? Burnout per hour of comprehension, against your own authoring at 1.0. Not reading speed — hours are hours. It says rebuilding a model you did not build draws harder on the same day. 2.4×
Burnout ratio ? LLM-path burnout divided by manual burnout. Above 1.0 means the assisted route costs you more than writing it by hand.
—
Unverified surface ? Share of the feature that is both unreviewed and unlike your own design — the part you cannot account for if it breaks at 3am.
—
Authoring Comprehension Untangling and rework
Bar length is clock time. Colour is what the time is spent doing.
—
A model, not a measurement. Every number here is an assumption you can move.
Ten years of DNS measurements reveal three trends across the Internet's most popular domains: email continues to consolidate around two providers, DMARC enforcement has hit a plateau, and a surprisingly large long tail of infrastructure defies easy classification.
Almost everything about how a domain handles email is sitting in public DNS, waiting to be counted. The MX record says where the mailbox lives. The SPF record says who may send on the domain's behalf. The DMARC record says what should happen when a message fails authentication. Put those three together for a million domains, every day, and you get something like a weather station for email infrastructure.
That is what I run. The pipeline takes the daily forward-DNS snapshots that the OpenINTEL project (University of Twente, SURFnet and SIDN Labs) publishes for the Tranco top-1M, and classifies each domain's MX hostname and SPF includes against open dictionaries of mailbox providers, sending platforms and SaaS applications. A typical day yields about 659,000 domains with MX records and 618,000 with SPF. OpenINTEL's archives make it possible to compute the same figures back to 2016, which turns a snapshot into a time series - and the time series is where things get interesting.
Three findings from the current data seem worth the community's attention.
In 2016, 44.6% of MX-publishing domains in the top million ran their own mail server. In the 18 July 2026 snapshot that figure is 22.4% - and it is still falling, down another half a percentage point in the last thirty days alone, which again seems worth the community's attention.
The domains didn't disappear; they moved. Google Workspace now receives mail for 21.8% of MX-publishing domains and Microsoft 365 for 16.8%. Together that is 38.6% of the measured Internet's inbound mail behind two companies. Nobody else comes close: the next named provider, Proofpoint, sits at 1.9%.
It is easy to read this as a market-share story, but for this community it is really a resilience story. The RIPE community has spent years discussing DNS and CDN centralisation; email is following the same path, just more quietly. When more than a third of popular domains depend on two providers to receive mail, an outage, a filtering change or a policy decision at either one propagates through the whole ecosystem at once. And unlike a CDN, email has no graceful fallback - a rejected message is simply gone.
There is a second-order effect too. The fewer independent operators there are, the more the remaining ones inherit the deliverability problems of a world tuned for the big two. Anyone who has tried to stand up a fresh Postfix box in 2026 and get its mail accepted at scale knows exactly what I mean.
458,467 domains in the current snapshot publish a DMARC record. On paper that is a success story a decade in the making. In practice, only 46.9% of those domains enforce anything - meaning p=quarantine or p=reject at pct=100. The majority publish a policy that asks receivers to do nothing.
What surprised me more than the level is the direction. The enforced share is not creeping upward; over the last thirty days it fell by 0.44 percentage points. The bulk-sender requirements that Google and Yahoo introduced in 2024 clearly drove publication - you can see the step in the adoption curve - but they set the bar at "have a DMARC record", and a very large part of the Internet stopped precisely there.
The records themselves tell the story better than any aggregate. The single most common DMARC record in the dataset, published verbatim by 58,064 domains, is:
v=DMARC1; p=none;Another 32,682 domains publish the same string minus the trailing semicolon, and thousands more publish minor byte-level variants of it. These are copy-pasted starter policies - created to satisfy a checklist, then never revisited. A p=none record with no rua= destination does not even collect the reports that would justify its own existence. It protects nobody; it just makes the adoption statistics look good.
Dictionary-based classification has a ceiling, and I want to be honest about where it is. Matching MX hostnames against ~310 provider patterns and SPF includes against dictionaries of ESPs, forwarders and gateways currently attributes about 81.5% of SPF includes and the large majority of MX records. What is left over is remarkable in its size: 36,455 unique MX hostnames that match no known provider, and tens of thousands of SPF include targets that appear on exactly one domain each.
Some of what surfaces in that tail is entertaining - 503 domains in the top million publish
localhost
as their MX, and 130 publish a literal
~
- but most of it is the unglamorous middle of the Internet: regional hosters, self-built Exim boxes, corporate gateways with vanity hostnames. This is precisely the population that deliverability research sees worst, because it is invisible to any measurement that only knows the big platforms. I publish the unmatched hosts openly with each daily run, partly as an invitation: if you recognise a hostname pattern, corrections land in the next day's snapshot.
The source is the daily OpenINTEL Tranco snapshot; pre-2022 history uses OpenINTEL's legacy Alexa top-1M source, which has a somewhat different composition. For each domain the primary MX (lowest preference) determines the mailbox provider; the apex SPF record determines senders; the
_dmarc
TXT record is parsed for policy, subdomain policy and pct. Aggregates, the full time series and the daily change-feed are published on the
project's stats page
; raw OpenINTEL data is deleted after each run per their data agreement.
The blind spots are worth stating plainly. Flattened SPF records - include chains replaced by raw IP ranges to duck the 10-lookup limit - hide the sending platform entirely. MX targets that are CNAMEs to a known provider are not unrolled, which pushes a small share of domains into "unknown". White-label deployments of Mimecast or Proofpoint are indistinguishable from self-hosting when the customer uses its own hostnames. And Tranco itself leans towards US and EU domains, so the picture is a picture of the popular Internet, not the whole one.
Ten years of these records tell one consistent story with three chapters: consolidation that shows no sign of slowing, an authentication standard that got adopted as a formality rather than a protection, and a long tail that resists being counted at all. Each chapter has a question attached. At what concentration does inbound mail become a systemic dependency worth the community's explicit attention? What would actually move DMARC from published to enforced, given that the 2024 mandates demonstrably did not? And how much of the Internet's mail infrastructure are we all failing to see because our dictionaries don't know its name?
I don't have firm answers. I do have the same measurement running again tomorrow at 23:00, and the day after that - which, over enough days, is how these questions tend to get answered.
The underlying DNS data comes from the OpenINTEL measurement platform of the University of Twente, SURFnet and SIDN Labs (van Rijswijk-Deij et al., IEEE JSAC 2016). Spotted a misclassified MX host or a missing provider pattern? Corrections are welcome and appear in the next daily snapshot.
Why some of us are still bothered about the way we write tests and what the “matcher” concept has to do with it.
During most of my Ruby career, I was that unpleasant person who honestly enjoys writing tests and is frequently concerned about the ways we write them.
This means treating unit tests like the rest of the codebase: like something that is supposed to be read by humans and something that should be written efficiently and expressively . Basically, like something that wouldn’t be boring and disgusting to read and write.
This also means that I find it useful, once in a while, to stop and reflect on why we write test code the way we write it. And can this be improved?
Let’s move the elephant in the room from our way at once: from my point of view, this way of thinking does not become obsolete due to AI agents, who “can write any number of tests without being bored.” If anything, short, readable, and expressive code means more in this age. I extend this argument a bit in the last section of the post.
So, thinking about “how do we write tests” leads to a mass of related, tightly intertwined questions: How does the typical test in the codebase look? How hard is it to write a new one? How hard is it to read and maintain an existing one? How does it affect the overall codebase maintainability?
And how the design of the test framework and its utilities affects all these considerations and is affected by them?
To understand how this way of thinking might be useful, let’s look at the lowest level of the test: just one check, or assertion.
Let’s perform a small “from the first principles” journey (bear with me!).
How do we check that the code we just wrote does what expected of it?
It starts easy when you have just a small amount of new code to test: one script, one utility function, one small class, things like that.
The first, most naive approach, is to just run the code, see what it outputs (prints to the console, renders in browser, or makes any other user-visible effect), and compare it visually with what you’d like it to output. Frequently, this is enough for a quick prototype or a throw-away script: just write the code, run it, say “aha” or “oh no,” tinker a bit till you are happy, and then move over.
Obviously, it becomes tiresome for any non-trivial code, or one that turned out to be not short-lived: you eventually need it to do more and more things under more and more circumstances, and just manually checking “my new case is working, and the old one is not broken” becomes a burden.
And so you need some kind of a “test script,” with “when we run it like this, that happens” codified. This “check what happens” should be easy to write, and it should provide useful feedback: “in this part of the test script, this assumption turned out to be incorrect.” This is what we frequently call “test assertion”, “test check,” or “test expectation,” depending on the context and tools used.
The API to assert things is one of the first services that any test tool provides. And one that, in my opinion, affects the test library usage and developer’s thinking process.
Of course, we can go to higher levels to think how we organize many tests and groups of tests. And also how do we run them – a lot of decisions can be made here: order of tests, their independence, running in parallel, rerunning only a subset. All of this unquestionably affects our thinking, the design of our tests, and the design of our software. But it all starts with one test – and one assertion.
Not everyone considers “how do we write one test” to be of any importance. In the “architecture-first” thinking, the particular code at the level of singular “paragraphs” and “phrases” – its brevity, expressiveness, or ease of modification – is frequently brushed off as insignificant. My way of thinking on ease of development and maintenance of software, though, gives this “low” level significance. I will follow this line of thinking for now without further argument (which I expressed many times already). And I ask you to be with me here, if only out of curiosity, “how some of us approach what they do.”
The simplest of such APIs is
assert(expression)
, with
expression
expected to return either truth/truthy value (the test passed) or false/falsy value (the test failed).
Frequently, this
assert
is even a part of the language itself, or its standard library – to be used as a debug or production guard against “impossible conditions.”
In testing, it might be used like this (usual “arrange, act, assert” structure) 1 :
arguments = prepare_arguments() # arrange
result = execute_code(arguments) # act
assert(result == expected_value) # assert
Here, only the last line has any calls that should be provided by a test library. Or, if it is a “core language” assertion feature, the only role of the test framework here is to provide a hook/handler for the signal that failed assertion produces (by raising an exception or other means).
Throw in some API or agreement how you put such fragments in separate tests and how are they executed (the common approach: every method/function in
tests/
folder files that is named
test_something
is run separately) – and this is already enough for the smallest, yet useful, “testing library.”
If not provided by the language itself, such an
assert
can be trivially implemented as a method that just throws an
AssertionError
exception if the passed argument is falsy. The exception’s backtrace will point to the failed line, giving enough basic information to debug. To make it a bit more friendly, a message argument can be added to the
assert
signature, allowing the developer to write:
assert(result == expected_value, 'Explanation of the case tested')
…and adding the explanation to the failure message.
In fact, the first JUnit library 2 was not much more than this.
But still, there was some more. Even in the most basic case – the “result should be equal to the expected outcome” – if the assertion fails, “it was not equal” is not enough useful information; “but what it was ” would be the immediate follow-up question. So the logical next step is to have a small utility wrapper:
assert_equal(result, expected_value)
…which compares two values and, if they aren’t equal, renders something informative, like
"expected: 1, was: 2"
.
A pedantic note: In JUnit, the declared order is actually
assert_equal(expected, actual). Modern JUnit and some of the JUnit-derived libraries preserve this order; others switch to(actual, expected); still others refuse to confine the developer and use neutral naming like(left, right). Finally, there are those that, like LuaUnit, make it a configuration option . While this might seem a “boring nuance,” we show that this order decision matters further in the article.
Once we have this helper function, one might think of other APIs in the same line of reasoning: assert that value is that of the expected type (and properly render what actual type it was otherwise); assert that it is a collection and has an expected number of items; assert that it includes some specific structural subpart, and so on.
These bunches of
assert_something
APIs are still, for all I can tell, the most popular testing API. All across the programming languages spectrum, the “xUnit-style” testing library is frequently the default/most used one, if not (in newer “batteries included” languages) part of the core distribution itself.
To work on this article (and, hopefully, the next parts), I made a private quick comparison document of many test libraries throughout the mentioned “spectrum” of modern programming languages. I am thinking about publishing it as a separate post/document, as it turns out to be of interest for any living soul other than me.
Still, another style exists, and it is almost equally widespread. And, as far as I am able to research, its popularity (if not the style itself) had originated from Ruby.
Around 2005-2007, there were a lot of blog posts ( here is one of the definitive ones) discussing the ideas of “behavior-driven development” – mostly, a new way of thinking, or rather a shift from the familiar ways. While the initial approach to test-driven development made the code author think in terms of “how can I write the test for my non-existent yet code, that will help me design it,” behavior-driven development suggests thinking in terms of “how can I describe the desired behavior of non-existent yet code.”
The difference might sometimes seem subtle, though it was believed that this shift of perspective might mean a lot.
One of the influential articles demonstrates how subtle the shift might seem: 2005’s
A new look at test-driven development
. Dave Astels makes a big distinction between “old” test-driven development with
assertEquals(expected, actual)
and “behavior-driven”
shouldBeEqual(actual, expected)
.
From today’s point of view, these two APIs might seem effectively indistinguishable. A lot of software developers today would frown at almost any syntax/API discussion as not making a difference for a “professional engineer.” But in those old times people considered that choice of singular “words” and the shape of a “phrase” in the programming language affects the writer’s thinking. I still believe this, and that this belief is one of the things that make me efficient.
In the article, Dave Astels also says that in Smalltalk, and possibly Ruby, it might be even more natural with (Smalltalk’s syntax):
actual shouldEqual: expectedorresult shouldBeNullor[2 / 0] shouldThrow: DivideByZeroException.
Soon the Ruby’s RSpec library was born , directly inspired by Dave’s article and with significant contributions of Dave himself. The first version provided almost exactly the same syntax Dave had described:
actual.should_equal expected
It went from several hard-coded
should_<something>
-methods like
should_equal
in 0.0.1, through metaprogrammed syntax variants like
should.do.something
in 0.0.4, then
should_do_something
(but now parsed into words metaprogrammatically), and, in version 0.8.0 (February 2007), introduced this
3
:
actual.should eq(expected)
Ruby’s flexible syntax allows omitting method call parentheses in unambiguous situations. The code above is an absolute equivalent of
actual.should( eq(expected) )…so
shouldhere is actually a method expecting one argument – a matcher . But it also allows omitting more parentheses and writing the same code soeqwould look like an operator between actual and expected:actual.should eq expectedThe meaning stays the same: the matcher object produced with
eq(expected)is a separate entity from.should.
These were the days when Ruby’s flexible syntax and the APIs it allowed to envision were inspiring other communities to try to implement something similar in their language. This has happened with Rails in general and its various APIs, this also has happened to RSpec. In some languages the idea was ported more or less straightforwardly, for others, it required stretching the syntax tricks just for the sake of mimicking the
actual should matcher
formula.
For a quick example, Python’s should-dsl had provided
actual |should| expectedby defining a specialshouldobject which had an operator|defined on it in a way that made the whole statement produce an expectation. I don’t think it is widely used.
Ruby has its classes open by design (you can continue the definition of any class – even a system one – in any program). And in the Ruby community in those days, “just throw what you need into the core class” was a popular development approach. So
.should
method was just added to every
Object
in the first RSpec versions.
Eventually, “extend every object just to write somewhat nicer code” fell out of popularity in the Ruby community, and in a few versions
RSpec settled
onto the “wrapper object” API, that didn’t require unconditional extending of the base
Object
:
expect(actual).to matcher
Here,
expect(actual)
creates a wrapper object with methods like
.to(matcher)
/
.to_not(matcher)
, which is almost as compact as
actual.should
, but doesn’t pollute every object in existence with test library’s methods.
The API akin to this “new” one, or RSpec’s initial
.should
syntax, can be nowadays found in many languages and testing libraries.
For example, Go’s
Gingko
uses
Expect(actual).To(Equal(expected))
which is exactly RSpec’s API, save for Go’s stricter punctuation
4
.
Both Scala and Kotlin enjoy their “infix function” notation for even greater punctuation flexibility than Ruby allows:
ScalaTest
with
actual should equal (expected)
and
KoTest
:
actual should eq expected
, respectively.
In JavaScript, two prominent libraries –
Jest
and
Chai
– both provide
expect
notation, but with a twist. In a bit weird turn of events, they both call their solution “matchers” while not providing separate “matcher objects”: it is
expect(actual).to.something(expected)
in Chai and
expect(actual).toSomething(expected)
in Jest. While visually similar to other RSpec-like libraries, this approach has a significant difference: “matchers” here are not separately constructed arguments, but methods of a wrapper object that
expect
produces. As a consequence, creating custom matchers (which we’ll discuss a bit later) requires extending that object, not just creating independent objects that correspond to matcher’s interface. Lua’s
Lust
follows the same road.
It is worth noticing that to use a matcher concept doesn’t necessarily require the whole formula with “should” / “expect to”. In fact, for all I can dig up through the developer thought archeology, the concept of matchers might’ve first emerged in the Java’s Hamcrest library – which even once became a part of JUnit 4 (but was later separated again), and then ported to many other languages .
Hamcrest just introduced the matcher concept into the familiar
assert<Something>
API with
assertThat
:
assertThat(actual, equalTo(expected))
Several other libraries follow this or similar structure, like:
EXPECT_THAT(actual, Eq(expected))
;
expect_that!(actual, eq(expected));
;
REQUIRE_THAT("Hello world", StartsWith("Hello") && EndsWith("world"))
;
Assert.That(phrase, Does.Contain("World"))
.
Considering the different choices of wording, we might say that the line is blurry here: whether, say, googletest should be considered “BDD-like”? If you squint out the punctuation, the difference of
expect(a).to(matcher(b))
vs
expect(a, matcher(b))
might be said to be “in the eye of the beholder.” And the same can be said about the importance (or lack of it) of the word choice: “assert” vs “expect” (vs “require” vs “should” vs …), for one writer, might shift the perception of their writing, and for another one, be just a familiar “sigil” they never much think of.
Finally, some testing libraries use the “matcher” concept even if it isn’t directly related to assertions, like C#’s mocking library Moq, that introduces matchers only to specify mocked method arguments.
This interruption won’t be long. I just want to remind you we are still here. I am in Ukraine, still serving in the army. Russia still tries to erase us.
In the last months, there was a glimmer of hope: while we are far from “winning,” at least some of the Ukrainian strategies seem to make the war continuation more and more painful for Russia.
In response to this (slight) change of the chances on the battlefield, Russia increased the barrages of ballistic missiles attacking our cities indiscriminately… And our dearest partners suddenly critically decreased the number of anti-missile munitions that allowed us to handle that. As if somebody really doesn’t like to see Russia having a slight chance to lose.
Oh, and one more thing: this summer, Russians increased deliberate targeting of the Ukrainian book industry (large book warehouse, printing houses and so on). Just in one strike on August 1, 8 million books were destroyed. I wonder what it says about the goals of the war. And who should see that. UPD: And while I was editing the final version, Russian missile burned the largest Kyiv book market overnight.
Let’s proceed with the rest of the article.
It is easy to dismiss the examples with matchers as “syntax/API nuances” which doesn’t change the general approach to writing tests. And, indeed, one can easily use matcher-enriched testing libraries to write tests in the exact style / order of
assert_equal
ones.
But when the concept of “assertion operator” and “matcher object” are separated, it becomes much easier (both mentally and technically) to construct new expressive checks by combining existing matchers or designing new ones.
In Ruby’s RSpec, matchers can be combined with logical operators and nesting.
Here are examples of the logical combination:
expect(user.email).to be_a(String).and match(/.+@.+/)
expect(user.occupation).to be_an(Occupation).or be_nil
Here,
be_a
,
match
,
be_nil
are separate matchers, and
matcher.and(matcher)
/
matcher.or(matcher)
produce new ones, which perform both checks and produce a clear resulting message.
Another way of the combination is nesting:
expect(emails).to all be_a String
# or, with all parentheses Ruby allows to omit:
expect(emails).to( all(be_a(String)) )
expect(response).to include(
name: starts_with('Admin'),
occupations: instance_of(Array),
badge: have_attributes(title: 'Hero')
)
…again,
all
,
be_a
,
include
,
starts_with
and so on are separate matchers, that can be put together to express a complicated
expectation
of the subject under test, in approximately as many words as the most straightforward human description.
Most of the test frameworks that use teh “matcher” concept explicitly support and encourage designing your own matchers. Usually, it would be an object with a simple interface (frequently, consisting of three methods:
match
,
failureMessage
, and
failureMessageWhenNegated
, or their equivalents), with their implementation being pretty trivial.
So, for example, in our large production Rails codebase we use more than a dozen of custom matchers (not even counting several third-party matching libraries). They allow us to write code like
expect { some_code }
.to create_record(User)
.with(name: 'Mary', email: 'mary@example.com')
# or:
expect(response).to be_successful(200...300).with_json(reloaded: true)
# or:
expect { some_code }
.to call_service(InternalService)
.with(some: 'arguments')
.returning(fake_value)
RSpec, in the style of the classic Ruby, even has a DSL to define matchers right beside the tests if the matchers are simple, so you can have some code like
# in the middle of some testing context,
# just "this local useful thing"
matcher :create_user do |username|
match do |response|
expect(response.status).to eq 201
expect(response.body).to include("Successfully created user #{username}")
end
end
# Usage for a subsequent several tests:
expect { some_call }.to create_user('alice')
…with the matcher itself being a two-liner, while still providing all the matcher services , like informative descriptions, backtraces pointing to right lines, and even diffs, when applicable.
Even before the arrival of AI-assisted coding, the tests were frequent victims of “nobody will read that” syndrome. The common reaction in many discussions of more expressive tests and utilizing the high-level language’s powers are full of monk-like resolve: tests should be “boring” and verbose, as if this is that unpleasant duty that only “true grown-up professionals” are able to perform (I go on a much longer and more heated rant on the topic… almost 10 years ago ; since than, I lost a lot of will to fight this particular fight with this much vigor, but kept the overall opinion on the matters).
So, to breathe in, breathe out summarize my rants, I firmly believe that we should strive for readable tests , and not in a “token by token, all tokens are easy to recognize” way, but in a “minimal amount of reading to give the idea clearly” way. In earlier writing I use the term “lucid” to distinguish “the meaning is conveyed efficiently” readability from “every particular phrase is easy to consume” readability.
Thinking in “matchers”, which are separate from the test/assertion itself, combinable and extendable, allows approaching the goal of “each test being a phrase that just describes what is tested” without resorting to high-level pseudo-languages of “Given/When/Then” (with a tar pit of implementation of every step being “somebody else’s problem”) 5 .
And the tsunami of AI coding agents doesn’t make it all irrelevant – quite the opposite. The code is now read much more frequently than before: by humans who control their agents (and frequently reading and an occasional nudge are all they do to achieve the final result), and by other agents, trying to get on board with the current work. In other words: more expressive code, including test code = smaller context window to keep in mind and fewer tokens to spend.
Clarity and precision are what matter more than ever. (Until we all are lost in waves of unreadable code goo which nobody even tries to open in the editor.)
I personally find it funny that it took the arrival of enormous and (objectively) inhumane language models to establish some principles that would’ve been hugely beneficial for humans many years ago. But until the “use your agent efficiently” discourse, everybody “was too busy” to actually write guidelines in simple no-BS markdown right beside the code, lead the work with clear descriptions, strive to keep the context obvious and short, endorse simple code-investigation tools like ast-grep , and, in general, make the codebases truly readable . More humane, if you will. God works in truly mysterious ways.
But whether you cater to human colleagues, your own sense of beauty, or more token-efficient and more controllable agentic development, clear and efficient tests might (just might, OK?) deserve a bit of your attention.
And, surprisingly for some, that niche, rarely-heard-of-anymore language, Ruby, and its libraries, give me several interesting angles to look at the topic. Some of them are even half-forgotten by modern Rubyists.
I hope to continue this train of thought, but (looks into the last two years of blog regularity) we’ll see.
…which should’ve been in the main flow, but it was too long already, so here is one aside observation about Python testing.
Speaking of approaches to a single assertion, pytest is an interesting case. It is one of the modern frameworks that actually seem to make
a step back
to having just a single
assert value
statement. The trick is that the usual conveniences like detailed reporting of what exactly was not equal to what is provided via metaprogramming, so if you run a test saying something like
x = 5
y = 6
assert x == y
…you’ll have an output saying:
AssertionError:
> assert x == y
E assert 5 == 6
…and for collections comparison it is even more detailed, able to render the detailed element-by-element diff. Such a “back to simple” approach can be found in, say, in
Nim
, which has a single main test function
check
.
But the approach is somewhat limited to the operators known to Pytest: even simple
assert re.match('^test$', 'tost')
fail will print
Assertion Failed: Assert None
6
instead of something like “expected ‘tost’ to much ‘^test$”.
Curiously, the library itself provides a “matcher-like thing” for floats comparison:
assert 0.1 + 0.2 == 0.3 # fails for obvious reasons
assert 0.1 + 0.2 == pytest.approx(0.3) # succeeds
The
pytest.approx
produces a special object that utilizes Python’s
==
implementation, which tries both values’
__eq__
method if the left is not compatible with the right and signals it with
NotImplemented
.
It is fairly similar to a matcher concept! However, there is only one such object in the library itself. There is at least one third-party
library
that implements many possible matchers with this trick (like
actual == AnyMatch('^admin:')
)
It is almost as if “matcher” as a base component of a testing library has some significant merit, so it might suddenly emerge on itself.
The French Ministry of the Economy and Finance has disclosed a data breach after an attacker accessed the General Directorate of Public Finances (DGFiP) systems and stole data belonging to 678,000 individuals.
This incident was discovered after a threat actor using the "ZeroBytes" handle claimed the attack and listed a stolen database for sale on August 12 on the PwnForums hacking forum.
"The in-depth investigations conducted since August 12, 2026, have established that, prior to their interruption, these access points had been used to consult and extract data concerning a total of 678,000 individuals and professionals, including tax data such as reference tax income, family quotient, and withholding tax rate, and, for businesses, data such as their company name and SIREN number," the French Finance Ministry said .
"Cadastral data relating to addresses and property sizes were also accessed. As soon as these data breaches were identified, the French Public Finances Directorate (DGFIP) notified the French Data Protection Authority (CNIL). The online accounts of individual and professional users were not compromised. User IDs and passwords were not compromised."
After detecting the attack, the French tax administration shut down access to sensitive information systems and continues investigating the incident with the help of the National Cybersecurity Agency of France (ANSSI) to assess the breach's full impact.
In a post on the hacking forum, ZeroBytes also claimed they gained access to the Serveur Professionnel de Données Cadastrales (SPDC), an online platform operated by the French tax authority that provides access to the country's central land registry and property ownership records.
While the portal gave them access to data on roughly 20 million French citizens, the threat actor claims they only managed to steal 252,149 records containing data on over 2 million people.
"We couldn't finish the extraction because honestly, it's just horrible to scrape and would have taken months. I'm still logged into the panel, so if you want, you can buy it along with the database," they said. "I'm not going to sell this one for very much anyway. And as always, no mention from France about this incident."
The French Finance Ministry added on Friday that it will contact all affected individuals starting next week via email or letter, with details on what data may have been accessed or stolen and the necessary precautions to take.
This is just the latest in a spree of cyberattacks and data breaches that have impacted multiple French government agencies in recent months.
In January, the French data protection authority fined the national employment agency France Travail €5 million after hackers stole the personal information of 43 million people. One month later, the French Ministry of Finance disclosed another data breach affecting over 1.2 million user accounts after hackers stole a database from the national bank account registry (FICOBA) systems.
More recently, France Titres, the government agency in France for issuing and managing administrative documents, also disclosed a data breach after a threat actor put up for sale a database containing 19 million records allegedly stolen from the National Agency for Secure Documents (ANTS).
Overall prevention scores can hide what happens after initial access. Once attackers are using valid credentials, prevention drops sharply.
The Blue Report 2026 measures defenses technique by technique across 338 million simulations run in customer production environments.
Encrypted hard drives protect data, including in the event of theft. Even if entire racks are carried out of the server room: without the correct passphrase for disk encryption, the hard drives are of little use. So of course we encrypt them. So far, so obvious.
This security, however, comes at a price. Every instance of human intervention during a reboot costs time and increases the risk of errors: if the boot process is interrupted, whether due to distraction or any imaginable incident, the server gets stuck at the password prompt and fails to boot up again.
We sometimes find ourselves needing to reboot servers more frequently. While a lot of software can be updated or reconfigured on the fly, this option stops at the Linux kernel: eventually, a reboot is required. That’s why we wanted a solution that allows us to reboot servers without the need for a manual decryption step.
Of course, one could try to automate our existing process, letting another computer with access to the passphrase connect to the rebooting server via SSH and decrypt it. However, this type of automation is tricky, because it turns the whole thing into a distributed system , with all the problems that entails. We wanted a solution in NixOS that does not depend on another computer, but retains the security of our existing solution. And that’s what we have achieved using kexec .
The kexec mechanism lets the kernel (k) execute another kernel (exec). Which means: The running kernel replaces itself with a new one without the actual server (i.e. the hardware) being shut down and restarted. Only the operating system restarts. The clever part: the old kernel can make information available in RAM that the new kernel can use. A passphrase for decrypting the hard drive, for example. As a nice side effect, we skip the firmware and bootloader parts of the reboot, saving several minutes of rebooting time, particularly on servers.
The LUKS passphrase could also be passed by simply writing it into a file before the reboot. But there’s a catch: During a full reboot, the system shuts down completely. For the newly starting kernel to be able to read this file, it would have to be stored unencrypted, which would undermine the security of our hard drive encryption. With kexec, the data can be passed via the server’s volatile RAM. This is a bit tricky though. In this article, we’ll show you how we implemented it.
You can’t have your cake and eat it, too? We simply didn’t want to accept this supposed wisdom for the matter of rebooting our servers. We want all of it: the security of full-disk encryption, of course, but also convenience, speed and reliable reboots. And if there’s one thing we don’t do, it’s giving up quickly just because something doesn’t work straight away.
So we set out to find a solution and came across two interesting sources with different approaches:
Both approaches have their pros and cons; neither was quite enough for us. That’s why we took the best bits from both and built our own solution.
We generate a temporarily valid passphrase. But we don’t stop there. Because you can manage multiple keyslots in LUKS. So we also create a keyslot, assign this temporary passphrase to it, and delete the keyslot immediately after a successful kexec. This ensures our encryption remains secure even if the temporary passphrase were to be leaked for any reason.
Even though the passphrase is invalidated immediately on the next boot, we want to take extra care to ensure that nobody can read it from the kernel command line, not even in the brief moment it takes us to invalidate the one-time passphrase. We therefore use the technique described in the blog post mentioned above : the one-time passphrase is embedded in a special initramdisk image, which is created specifically just before the kexec reboot.
And this is our specific solution: We extend
systemd.services."prepare-kexec"
with the following steps:
Immediately after mounting the root filesystem, we remove the key slot via a custom systemd unit defined in
boot.initrd.systemd.services
.
Today, our reboot no longer requires any manual intervention. And it now takes just over 2 minutes. A simple
systemctl start kexec.target
is all that’s needed. Whether triggered manually or automatically, the server comes back up fully functional and ready for use.
Would you like our complete implementation as ready-to-use code? You can find our code in a sample configuration below this article. Enjoy!
Sample NixOS module:
{ config, lib, pkgs, ...}:
let
luksDevice = config.boot.initrd.luks.devices."yourdevice".device;
in
{
# Concept:
# To reboot a server using kexec, we need to alter the nixos
# prepare-kexec script, because we use full disk encryption.
# We add a temporary, random key to LUKS keyslot 31. This key is
# also added to the init ram disk image.
# We have to set the keyfile and fallbackToPassword option in the
# luks.device."yourdevice".If the keyfile exists it will be used to
# decrypt the disk, if not it will ask for the passphrase.
# We immediately delete the LUKS slot after mounting.
systemd.services."prepare-kexec" = {
path = with pkgs; [ cpio cryptsetup gzip ];
script = lib.mkForce ''
set -euo pipefail
umask 0077
# Don't load the current system profile if we already have a
# kernel loaded.
if [[ 1 = "$(</sys/kernel/kexec_loaded)" ]] ; then
echo "kexec kernel has already been loaded, prepare-kexec skipped"
exit 0
fi
p=$(readlink -f /nix/var/nix/profiles/system)
if ! [[ -d $p ]]; then
echo "Could not find system profile for prepare-kexec"
exit 1
fi
if ! [[ -f "/LUKS-Passphrase-file.txt" ]]; then
echo "Could not find luks-passphrase file"
exit 1
fi
# add 256 random bytes temp key to the LUKS keyslot 31
TEMP_DIR="$(mktemp -d --tmpdir=/dev/shm)"
mkdir "$TEMP_DIR/etc"
head -c 256 /dev/urandom>"$TEMP_DIR/etc/tmp-passphrase"
cryptsetup luksAddKey --batch-mode --key-slot 31 ${luksDevice} "$TEMP_DIR/etc/tmp-passphrase"</LUKS-Passphrase-file.txt
# create a new cpio archive and append it to the original initrd
cd "$TEMP_DIR"
cp "$p/initrd" "$TEMP_DIR/initrd.img"
find etc | cpio -H newc -o | gzip >> "$TEMP_DIR/initrd.img"
# load the kernel with the new initrd
kexec --load "$p/kernel" --initrd="$TEMP_DIR/initrd.img" --append="$(cat "$p/kernel-params") init=$p/init"
'';
};
boot = {
initrd = {
luks.devices."yourdevice" = {
fallbackToPassword = true;
keyFile = "/etc/tmp-passphrase";
};
systemd.services.clear-luks-keyslot = {
description = "Clear the LUKS key slot after successful kexec";
wantedBy = [ "initrd.target" ];
after = [ "systemd-cryptsetup@root.service" ];
serviceConfig.Type = "oneshot";
path = with pkgs; [ cryptsetup ];
script = "cryptsetup luksKillSlot --batch-mode ${luksDevice} 31 || true";
postMountCommands = ''
${pkgs.cryptsetup}/bin/cryptsetup luksKillSlot --batch-mode ${luksDevice} 31 || true
'';
};
};
}
Relief from the unrelenting heat is finally arriving this week across western Europe , with more unsettled conditions bringing the chance of rain. Temperatures are expected to fall back towards the seasonal norm, with some parts of the UK even falling about 3C below normal by Thursday. After such a scorching summer, and with a temperature drop of more than 15C compared with last Thursday’s heat, it may feel surprisingly chilly.
There are signs that some substantial rain could be on the cards for the south of the UK this week as a cold front moves in from the Atlantic on Monday night into Tuesday. As of Monday, parts of southern England had endured 62 consecutive days of no rainfall – this is observed at a Met Office weather station in Wisley, Surrey. Even the last recorded rainfall here on 16 June amounted to just 0.4mm.
The more interesting weather arrives from Wednesday into the weekend, as low pressure moves across the UK, bringing frequent sharp showers and potentially thundery downpours.
After such a prolonged dry spell, even a few substantial bursts of rain will make a noticeable difference to the environment. Continental Europe is also likely to turn wetter as the main cold front sinks southwards into France and Germany on Wednesday. Farther south, however, rainfall is likely to be more hit and miss, relying heavily on showers and thunderstorms, with this risk really rising from Thursday.
While the heat is subsiding on this side of the Atlantic, the southern US states have extended extreme heat warnings this week. There is a significant lack of overnight cooling and relief from the heat, with overnight lows not falling much below 27-28C (low 80sF). For perspective, the UK’s highest overnight minimum during the late June heatwave reached about 22-23C.
High humidity is also contributing to the uncomfortable conditions and is helping push the heat index to such high levels to warrant the extreme heat warnings. The combination of air temperature and humidity produces the heat index, or apparent temperature, which better reflects how hot conditions feel. Apparent temperatures of 31-33C are being modelled as the overnight lows, with the National Weather Service warning of “dangerously hot conditions” as the forecasted daytime heat index values could exceed 46C, especially across Louisiana and Mississippi.
On Friday, Microsoft confirmed it has begun working on a security patch for a Defender zero-day vulnerability named "ShieldBreak."
A security researcher who uses the "Nightmare Eclipse" handle disclosed this privilege escalation vulnerability after Microsoft released the August 2026 Patch Tuesday security updates.
"Microsoft is aware of the reported vulnerability and is actively investigating the validity and potential applicability of these claims," a Microsoft spokesperson told BleepingComputer when asked for a statement regarding the new ShieldBreak zero-day.
"Microsoft is committed to investigating security issues and updating impacted products to protect customers as soon as possible."
Nightmare Eclipse described ShieldBreak as a bypass for RoguePlanet , another Defender privilege escalation flaw disclosed in June, and shared a ShieldBreak proof-of-concept (PoC) exploit that local attackers with limited permissions can use to gain SYSTEM privileges on fully patched Windows 10, Windows 11, and Windows Server systems.
"Microsoft has failed to properly patch the RoguePlanet vulnerability CVE-2026-50656, this PoC demonstrates a full patch bypass," Nightmare Eclipse said .
"The PoC was tested in the latest version of windows 11 25h2 (+Canary channel) and windows server 2025, the PoC also have a 100% success rate. Please note that Windows 10 (and respective server editions) are not currently supported, they are however vulnerable to ShieldBreak as well."
Vulnerability analyst Will Dormann confirmed last week that the ShieldBreak exploit works but added that Microsoft Defender must also be enabled for attackers to escalate privileges.
On Friday, three days after ShieldBreak was disclosed, Microsoft said it's now tracking the flaw as CVE-2026-69414 and confirmed it's working on a patch, but has yet to acknowledge that Nightmare Eclipse found it.
"Microsoft is aware of an elevation of privilege in the Microsoft Malware Protection Engine in Microsoft Defender publicly referred to as 'ShieldBreak,'" the company said. "We are working to provide a high quality security update that addresses this vulnerability. We will provide information in this CVE when the update is available."
Nightmare Eclipse publicly disclosed ShieldBreak without notice to Microsoft as part of an ongoing dispute with the company over its vulnerability disclosure and bug bounty practices.
Days after the researcher published PoC exploits without prior notice, Microsoft responded with warnings of legal action against people engaging in "malicious activity causing real harm" to its customers, prompting many to believe that the company was directly threatening the security researcher.
Since April, Nightmare Eclipse has disclosed multiple zero-day exploits targeting Microsoft Defender, BitLocker, and various other Windows components, now known as LegacyHive , RoguePlanet , BlueHammer , RedSun , YellowKey , GreenPlasma , MiniPlasma , and UnDefend .
While the company fixed the YellowKey, GreenPlasma, and MiniPlasma flaws as part of the June 2026 Patch Tuesday and RoguePlanet in July , the other security flaws disclosed by Nightmare Eclipse remain zero-days and are still awaiting an official patch.
Overall prevention scores can hide what happens after initial access. Once attackers are using valid credentials, prevention drops sharply.
The Blue Report 2026 measures defenses technique by technique across 338 million simulations run in customer production environments.
PLATFORMS
Every solid desktop background color shipped by classic operating systems and desktop environments.
#008080
Windows 95
1995 · Windows
Teal defined the era — the first face millions saw at boot.
#3a6ea5
Windows Me
2000 · Windows
The last of the 9x line — dressed in Windows Classic blue.
#008080
Windows NT 4.0
1996 · Windows
The NT kernel meets the Windows 95 shell — teal desktop and all.
#c0c0c0
Windows 3.0
1990 · Windows
The release that made Windows a success — and let you recolor the whole desktop.
#0055aa
Amiga Workbench 1.x
1985 · Amiga
Four-color glory: the unmistakable Workbench palette.
#3a6ea5
Windows 2000
2000 · Windows
A polished blue-gray palette that bridged two eras.
#c0c0c0
Windows 3.1
1992 · Windows
The desktop that taught the world to point and click — with a scheme for every taste.
#008080
Windows 98
1998 · Windows
The familiar teal desktop, now folded into the web.
#808080
Xfce
1996 · Desktop Env.
A lightweight Unix desktop with a broad, practical palette.
#336698
BeOS
1995 · BeOS
The multimedia OS with a calm, steady blue desktop.
#574c8f
Solaris 9
2002 · Solaris
CDE backdrops and the violet field behind an iconic Solaris watermark.
#00c000
BleskOS
2020 · BleskOS
A tiny hobby OS with a boldly green desktop.
#b3b3da
Mac OS 8
1997 · Mac OS
Platinum-era desktop patterns in soft, muted accents.
#336698
Haiku
2009 · Haiku
The open-source heir to BeOS — same focused desktop, same signature blue.
#21578d
ReactOS 0.4.x
2016 · ReactOS
The open-source Windows, rebuilt from nothing — classic desktop, real colors.
#505050
SerenityOS
2018 · SerenityOS
A love letter to '90s desktops — built from scratch, themed to the teeth.
#57ff81
Windows 1.0
1985 · Windows
The first Windows — a desktop dithered from two colors to fake a third.
#57ffff
Windows 2.0
1987 · Windows
Dithering, abandoned: Windows 2.0 just used the cyan straight.
#aaaaaa
Amiga Workbench 2.0
1990 · Amiga
The gray refresh: Workbench 2.0 traded blue for a cool 3D gray.
#5454d4
FreeGEM
1999 · GEM
GEM lives on — a desktop dithered from gray and blue.
#1d99f3
KDE Plasma 6
2024 · Desktop Env.
The modern, endlessly tweakable Linux desktop — anchored by Breeze blue.
#008080
Windows NT 3.x
1993 · Windows
The first Windows NT — a 32-bit workstation dressed in Program Manager teal.
The goal of in-place initialization is to enable the construction of types directly into a memory location without any additional moves or copies. When working with big types this can be more efficient and even prevent stack overflows. But some types are what we call address sensitive and so cannot be moved for correctness reasons.
There is some disagreement about how we should encode in-place initialization in the language. There are conflicting requirements and constraints at play, and reconciling those is tricky. I believe that the right way to attack the problem space is not by introducing a single feature, but by introducing a 4-level feature hierarchy for in-place initialization .
At the lowest level we have raw pointers and
MaybeUninit
. This is by far the
most flexible way to encode emplacement, but it comes at the cost of virtually
everything else. This is both how the
pin-init
crate and
placing
crate are
implemented internally. To read more about this see my post on
placing
functions
where I work through
a full desugaring.
The way I categorize this level is as: “It’s better than nothing”. It’s good
that we have
some
way to encode emplacement in the ecosystem today, even if it
leaves much to be desired. Here is a basic example using
MaybeUninit
, raw
pointers, and
unsafe
to defer initialization:
use std:: mem:: MaybeUninit ;
let mut x = MaybeUninit ::< A >:: uninit (); // 1. Create an uninit place `x` of type `A`
let y: * mut A = x. as_mut_ptr (); // 2. Take a raw pointer `y` to `x`
unsafe { y. write ( A { .. }) }; // 3. Initialize all fields through `y`
let mut x = unsafe { x. assume_init () }; // 5. Notarize `x` as initialized
let y: & mut A = & mut x; // 6. `x` is initialized and can be used as normal
On step 5 we do move the value of
x
. If wanted to notarize
x
as initialized
without moving it, we would need to call
MaybeUninit::assume_init_mut
, but
this returns an
&mut T
rather than change
T
in-place. Without additional
language features, it’s impossible to notarize an owned value as initialized
without moving it or turning it into a reference.
Raw pointers are very powerful, but the compiler cannot check their correctness which places an additional burden on the programmer. What we need is an abstraction that can encode most of what raw pointers can, but in a way that the compiler can statically check it within a reasonable amount of time.
My preferred proposal for this is Ding Xiang Fei’s
&uninit
/
&own
reference
pair, but there are more proposals that could fill this slot. The idea of
&uninit
/
&own
that we can take an
&uninit
reference to a type, and once all
of its fields have been initialized can then be
notarized
into an
&own
reference. There are four steps to the process here:
x
of type
A
.
&uninit
reference to
x
.
&own
reference.
let x: A ; // 1. Create an uninit place `x` of type `A`
let y: & uninit A = & x; // 2. Take an `&uninit` reference `y` to `x`
* y = A { .. }; // 3. Initialize all fields of `y`
let y: & own A = y; // 4. The reference `y` is `&own` from here on out
x = y; // 5. Notarize `x` as initialized
let y: & mut A = & mut x; // 6. `x` is now initialized and can be used as normal
The main innovation of this proposal is that it makes uninitialized places a
first-class thing we can talk about and reference. The example above can already
be written today without
&uninit
and
&own
by writing
let a; a = A { ... };
.
But this doesn’t work across functions, which is something we can do with
&uninit
/
&own
:
// Convert an `&uninit A` into an `&own A`.
fn init_a < ' a >( y : &' a uninit A ) -> &' a own A {
* y = A { ... };
y
}
let x: A ; // 1. Create an uninit place `x` of type `A`
x = init_a ( & x); // 2. Initialize `x`
let y: & mut A = & mut x; // 3. `x` is now initialized and can be used as normal This is not a simple feature, but it’s not a simple problem either. This makes uninitialized values both first-class and safe to pass around and initialize. By design it wants to be as expressive as possible, which means prioritizing control above all else.
Where references prioritize control, placing functions prioritize ergonomics.
Placing functions
are
functions which re-write the
return
keyword to write data to an out-pointer
rather than copying. It can be implemented in terms of either raw pointers or
&uninit
/
&own
references. But unlike either of those features it doesn’t
require any further changes to the function signature.
To show where this is useful we need to think about how we would transition
existing code to emplace. Here is a typical function which returns a value of
type
A
, and assigns it to the variable
x
.
// Create a value of type `A`
fn init_a () -> A {
A { ... }
}
let x = init_a (); // 1. Create a value of type `A`
If you compare this to the in-place init example using
&uninit
and
&own
,
you’ll notice just how much simpler this is. No fancy references, lifetimes, and
notarization. But unfortunately it also copies, which if
A
contains many
fields might be a problem. So ideally we’d have something that can emplace but
without all the ceremony:
// Create a value of type `A` in-place
# [ emplace ]
fn init_a () -> A {
A { ... }
}
let x = init_a (); // 1. Create a value of type `A` in-place
Not bad, right? Of course this isn’t as flexible as
&uninit
+
&own
. But for
the common cases this should be plenty. Though we probably don’t just want this
to be a one-off attribute, but probably its own keyword. My current thinking is
that we should encode this as an effect like
const
, and expose all effects using
the
with
keyword
:
// Create a value of type `A` in-place
fn init_a () -> A with emplace {
A { ... }
}
let x = init_a (); // 1. Create a value of type `A` in-place
A function annotated with the
emplace
effect guarantees that it will write
its return value to an out-pointer rather than copy it. That makes it so these functions can “return”
!Move
types
, which is a requirement for
safe constructors of unconditionally self-referential types
.
In RFC 3943 Amanieu is proposing the addition of MIR Move Eliminations . This enables the compiler to automatically eliminate moves at the MIR level as an optimization, which could apply to some of the examples we’ve looked at previously:
// Create a value of type `A`
fn init_a () -> A {
A { ... } // assume `A` is 2kB in size
}
let x = init_a (); // 1. The optimizer ensures `x` is created in-place.
This looks very similar to the “placing functions” proposal, but encoded as an
optimization. Optimizations should only affect the performance of a program and
not the semantics, and so cannot be relied on. That means that for example
returning
!Move
types from a MIR-move eliminated function would still be
disallowed, since the elimination is an implementation detail of the compiler
and not a part of the language.
If some novel behavior is important for the correctness of the language, it should be surfaced in the notation. Even if we could guarantee that certain expressions always emplace, as long as there are exceptions, then before long we’re finding ourselves explaining the differences between lvalues, rvalues, prvalues, glvalues, and xvalues 1 . I think it’s much better to write out requirements in code than expect programmers to infer them from context clues.
Perhaps there is a future where Rust can guarantee that every single expression
in every single location can guarantee emplacement. At that point a notation to
opt-in to emplacement would be superfluous, and we might choose to make that the
default behavior of expressions over an edition
1
. But we can’t do this
with a single feature straight away, so it’s better to start with two
complementary features, which is not unlike what we’ve done with
const
.
We want in-place initialization so we can work with self-referential types, which require that they aren’t moved in memory. As well as to improve the overall performance of the language, by eliminating copies. Here are the four levels of in-place initialization features we’ve discussed in this post:
| Expressivity | Memory | Signature | Semantics | |
|---|---|---|---|---|
| 0. Raw pointers | Expansive | Unsafe | Changed | Guaranteed |
| 1. References | Expansive | Safe | Changed | Guaranteed |
| 2. Placing fns | Basic | Safe | Unchanged | Guaranteed |
| 3. MIR move elimination | Basic | Safe | Unchanged | Not guaranteed |
Rust tries to balance ergonomics with expressivity and control. We have pointers in the language today, and the compiler should always optimize whatever it can. By splitting the safe abstractions into a pair of high-level and low-level features we can ensure that the easy cases feel easy, but control is still there for those that need it.
The System and Service Manager systemd provides the building blocks for Linux systems, and it’s the default in NixOS — though, there are a few alternatives such as finix and sixos that currently explore this design space.
As someone currently self-hosting a few services on my NixOS servers, it’s become a constant in my toolkit.
Current status of my servers, as generated by
nix-topology
(with a few custom patches) ❄️
Recently, I’ve been doing some upgrades and maintenance in one of my servers running Immich , and I thought I would check the current job schedule to ensure they are configured correctly in my timezone.
[admin@metropolis:~]$ systemctl list-timers --all
NEXT LEFT LAST PASSED UNIT ACTIVATES
Sat 2026-08-15 10:44:47 CEST 15min Sat 2026-08-15 10:14:47 CEST 14min ago disk-monitor.timer disk-monitor.service
Sat 2026-08-15 11:00:00 CEST 30min Sat 2026-08-15 10:00:08 CEST 29min ago logrotate.timer logrotate.service
Sat 2026-08-15 15:21:37 CEST 4h 52min Fri 2026-08-14 15:21:40 CEST 19h ago acme-renew-immich.gvolpe.com.timer acme-order-renew-immich.gvolpe.com.service
Sat 2026-08-15 22:52:43 CEST 12h Fri 2026-08-14 22:52:43 CEST 11h ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Sun 2026-08-16 00:00:00 CEST 13h Sat 2026-08-15 00:00:08 CEST 10h ago custom-immich-stop-trigger.timer custom-immich-stop-trigger.service
Sun 2026-08-16 00:00:01 CEST 13h Sat 2026-08-15 00:00:08 CEST 10h ago postgresql-backup.timer postgresql-backup.service
Sun 2026-08-16 00:00:05 CEST 13h Sat 2026-08-15 00:00:08 CEST 10h ago borgbackup-job-immich-s3-media.timer borgbackup-job-immich-s3-media.service
Sun 2026-08-16 00:00:05 CEST 13h Sat 2026-08-15 00:00:08 CEST 10h ago borgbackup-job-postgres.timer borgbackup-job-postgres.service
Sun 2026-08-16 00:00:30 CEST 13h Sat 2026-08-15 00:00:35 CEST 10h ago s3-replica.timer s3-replica.service
Sun 2026-08-16 03:00:00 CEST 16h Sat 2026-08-15 03:00:08 CEST 7h ago s3ql-fsck.timer s3ql-fsck.service
Sun 2026-08-16 03:30:00 CEST 17h Sat 2026-08-15 03:30:06 CEST 6h ago custom-immich-start-trigger.timer custom-immich-start-trigger.service
Mon 2026-08-17 00:00:00 CEST 1 day 13h Mon 2026-08-10 02:00:08 CEST 5 days ago nix-gc.timer nix-gc.service
Mon 2026-08-17 01:39:34 CEST 1 day 15h Mon 2026-08-10 02:52:22 CEST 5 days ago fstrim.timer fstrim.service
13 timers listed.
As the number of jobs grows, it’s easy to lose track.
However, looking at this information has got me wondering whether it was possible to get this information directly from my NixOS configuration instead of having to deploy before checking — i.e. build-time checks instead of runtime checks.
A nice property of NixOS is that it produces a derivation at build time, which is a build recipe of the entire system that can be easily inspected; and the systemd configuration that’s part of it is no exception.
nix-repl> nixosConfigurations.metropolis.config.systemd.timers
{
"acme-renew-immich.gvolpe.com" = { ... };
borgbackup-job-immich-s3-media = { ... };
borgbackup-job-postgres = { ... };
custom-immich-start-trigger = { ... };
custom-immich-stop-trigger = { ... };
disk-monitor = { ... };
fstrim = { ... };
logrotate = { ... };
nix-gc = { ... };
postgresql-backup = { ... };
s3-replica = { ... };
s3ql-fsck = { ... };
}
nix-repl> nixosConfigurations.metropolis.config.systemd.timers.s3-replica.timerConfig
{
OnCalendar = "*-*-* 0:00:30";
Persistent = false;
}
So the information is available and we can process it however we see fit. In my case, I wanted to create a Mermaid flow diagram with each server’s schedule, and here’s the result (see it on Mermaid Live ).
This was hacked-up together with a Python script that fetches the core information with the following command:
nix eval --json .#nixosConfigurations.metropolis.config.systemd.timers \
--apply 'builtins.mapAttrs (name: timer: timer.timerConfig)' | jq
Which produces the following JSON data:
{
"acme-renew-immich.gvolpe.com": {
"AccuracySec": "86400s",
"FixedRandomDelay": true,
"OnCalendar": "daily",
"Persistent": "yes",
"RandomizedDelaySec": "24h",
"Unit": "acme-order-renew-immich.gvolpe.com.service"
},
"borgbackup-job-immich-s3-media": {
"OnCalendar": "*-*-* 0:00:05",
"Persistent": false
},
"borgbackup-job-postgres": {
"OnCalendar": "*-*-* 0:00:05",
"Persistent": false
},
"custom-immich-start-trigger": {
"OnCalendar": "*-*-* 03:30:00",
"Unit": "custom-immich-start-trigger.service"
},
"custom-immich-stop-trigger": {
"OnCalendar": "*-*-* 00:00:00",
"Unit": "custom-immich-stop-trigger.service"
},
"disk-monitor": {
"OnBootSec": "5m",
"OnUnitActiveSec": "30m"
},
"fstrim": {
"OnCalendar": [
"",
"weekly"
]
},
"logrotate": {
"OnCalendar": [
"hourly"
]
},
"nix-gc": {
"OnCalendar": [
"weekly"
],
"Persistent": true,
"RandomizedDelaySec": "0"
},
"postgresql-backup": {
"OnCalendar": "*-*-* 0:00:01",
"Unit": "postgresql-backup.service"
},
"s3-replica": {
"OnCalendar": "*-*-* 0:00:30",
"Persistent": false
},
"s3ql-fsck": {
"OnCalendar": "*-*-* 03:00:00",
"Persistent": false,
"Unit": "s3ql-fsck.service"
}
}
Then it’s all about processing this data to generate the Mermaid diagram in the desired format.
The script follows this order of evaluation, and it defaults to UTC if both values are absent.
nix-repl> nixosConfigurations.metropolis.config.systemd.globalEnvironment.TZ
"Europe/Warsaw"
nix-repl> nixosConfigurations.metropolis.config.time.timeZone
"Europe/Warsaw"
Multiple timezones can be set individually on each service unit, but this is not a feature I need right now.
On a running server, we can generate a diagram of systemd dependencies with the following command:
systemd-analyze dot --require --from-pattern="*.service" --to-pattern="*.service" \
| dot -Tsvg > services.svg
Which renders something like this for this NixOS server:
That can quickly get pretty unreadable, so in general it’s more useful to analyze specific units and not all of them, e.g.
[admin@metropolis:~]$ systemctl show immich-server.service -p Requires -p Wants
Requires=postgresql.target s3ql-mount.service -.mount system-immich.slice
Wants=-.mount
These tools are useful for our daily server maintenance tasks, but can we verify dependencies without actually deploying first?
It is no surprise that this information is available in our NixOS configuration blueprint as well.
nix-repl> nixosConfigurations.metropolis.config.systemd.services.s3 <TAB>
nixosConfigurations.metropolis.config.systemd.services.s3-aliases
nixosConfigurations.metropolis.config.systemd.services.s3-replica
nixosConfigurations.metropolis.config.systemd.services.s3ql-auth
nixosConfigurations.metropolis.config.systemd.services.s3ql-fs
nixosConfigurations.metropolis.config.systemd.services.s3ql-fsck
nixosConfigurations.metropolis.config.systemd.services.s3ql-mount
The following one-liner command shows how to get all systemd dependencies directly from our NixOS configuration:
nix eval --json .#nixosConfigurations.metropolis.config.systemd.services \
--apply 'builtins.mapAttrs (name: service: { inherit (service) requires wants wantedBy requiredBy; })' \
| jq 'with_entries(.value |= with_entries(select(.value != []))) | with_entries(select(.value != {}))'
To keep it brief, here’s part of the JSON data that’s produced by it:
{
"immich-machine-learning": {
"requires": [
"postgresql.target",
"s3ql-mount.service"
],
"wantedBy": [
"multi-user.target"
]
},
"immich-server": {
"requires": [
"postgresql.target",
"s3ql-mount.service"
],
"wantedBy": [
"multi-user.target"
]
},
"nginx": {
"wantedBy": [
"multi-user.target"
],
"wants": [
"acme-immich.gvolpe.com.service"
]
}
}
I didn’t create any diagrams out of this, but we have the necessary data if that’s something we want to do.
I thought this was worth sharing, as NixOS doesn’t cease to amaze me 🤩. Safe to say I’m always learning new things, and more importantly, having tons of fun running these servers! Can’t recommend it enough.
What do you think? Let me know in the comments below 👇
Best, Gabriel.
Open Digital Infrastructure represents the set of open-source code, standards and knowledge assets that digital building blocks like software libraries, compilers, communication or network protocols are composed of.
They are created by individuals, volunteer communities, in research institutions and SMEs or other corporate environments. Together, they form a foundation of free and public code that is designed to solve common challenges – firstly, in programming, but when applied, also to provide a multitude of core functions for society.
Comments (0) Trackbacks (0) Leave a comment Trackback