Controlled experiment demonstrates how open models can bypass DNA sequence filtering — leading to industry-wide update of filtering software; goal: thwart bioterrorism before it starts
Microsoft recently announced That it used artificial intelligence to develop 75,000 new versions of toxins – proteins that harm the body. If that scares you, you should know that Microsoft actually claims that this is a positive development. And I agree with it. But to understand why, and how this innovation might save us from a massive man-made terrorist attack, you have to get down to the details. And maybe even go back a little to history.
The biological attack at the White House
In late 2023, an extraordinary envelope was received at the White House. Extraordinary in the sense that it could have killed a great many people. Inside the envelope was a short anonymous letter, and next to it a small test tube. The letter issued an ultimatum to the United States Department of Transportation, stating that if working conditions for truck drivers worsened, the writer would “turn the capital into a ghost town.”
These kinds of threats aren't as rare as we'd like to think, but the test showed that the writer might have the power to carry out the threat. The tiny container contained a white powder called ricin: a toxic protein extracted from plants. One gram of the substance is enough to kill About four thousand people, if it manages to reach their respiratory tract. To date, there is no antidote or cure for the venom, and even survivors are often left with Long-term damage to their internal organs.
It's understandable why the U.S. Secret Service was under a bit of pressure over the letter. Health authorities were bracing for ricin poisonings across the United States, and the fear only grew as Doomsday approached—the date when new traffic laws would go into effect—without the sender of the letter having been identified or caught yet.
On the fourth of January, 2024, The new laws have been passed.. Government officials held their breath—literally, because ricin is extremely toxic when inhaled—but there were no mass ricin poisonings that year. Washington, D.C., did not become a ghost town, at least as of this writing.
But the security services know that it's only a matter of time.
The unpleasant truth is that we are surrounded by creatures capable of producing toxins: from bacteria to plants. In fact, the ricin protein is produced in the seeds of the castor bean plant, which is widespread throughout the world. And that one of its nuclei can kill a childA skilled genetic engineer could relatively easily insert the genetic sequence that codes for ricin production into other plants, or perhaps even bacteria, and produce entire containers of ricin for any purpose. And if he were even more ambitious, he could expand to other toxins from the animal world.
Scary? Absolutely. But the people responsible for our security are not incompetent, for the most part. Governments understood the magnitude of the threat as early as the 1990s, and acted to reduce the danger.
What did they do? They put themselves in the shoes of the average murderous genetic engineer, and realized that he would try to order DNA sequences from the companies that produce them. What sequences? Of course – the ones that, when incorporated into plants, would lead to the production of ricin in large quantities.
Fortunately, because the structure of ricin is very distinct and precise, the relevant DNA sequences can be easily identified. Thus, every company that currently produces DNA automatically checks the sequences that are ordered from it to ensure that they cannot be used to produce ricin, or a multitude of other toxins and risk factors.
Then artificial intelligence entered the business.
The protein engine
Proteins are made up of chains of amino acids. If the words aren't familiar to you, just think of each protein as if it were made of a string of beads. Some beads are magnetic, and attract each other. Others repel each other. Some can only sit comfortably on a few other beads. Some beads are strong and stable, able to withstand the pull of many others, and some are weak beads that will crumble under the pressure. Some proteins are made up of just a few dozen of these beads. Others are made up of chains of tens of thousands of beads.
When you take this chain and put it in a cell, magic happens: the beads pull and push each other, connect and disconnect and reconnect. In a short time, the chain curls and folds back on itself, forming a three-dimensional structure. This is the finished protein – a tiny machine – that can now perform tasks in the cell.
The ricin protein is made up of 529 such beads. Each bead must be in its exact place in the chain for it to fold correctly, resulting in the final toxic protein.
But what if we replaced one of the beads on the original chain?
If you had asked this question five years ago, the response from the scientific community would probably have been a collective shrug. It was very difficult – and in many cases impossible – to figure out how changes in the bead chain would affect the final structure of the protein.
Then came the turn of artificial intelligence.
In 2021 The AlphaFold-2 artificial intelligence engine was demonstrated, which has been described as “revolutionary,” and rightly so. Its creators, by the way, won the Nobel Prize in Chemistry in 2024. Again, rightly so. This engine has allowed scientists around the world to understand how changes in the chains of beads can create new proteins. In other words, how to build tiny machines that can affect cells from the inside.
Back when AlphaFold was first demonstrated, I wrote about this new engine And how it will change the worldI predicted that it would allow scientists to develop new drugs easily and efficiently, and in the same breath I warned that it would also provide terrorists with tools to develop new harmful substances.
Microsoft researchers are also aware of the risks, and recently decided to test how prepared human society is for them. How? Simple: They used artificial intelligence to produce thousands of variations of ricin – each with a minor change in the protein's bead chain, but of a kind that should leave the overall structure of the protein as it was originally. Then they figured out which DNA sequences needed to be ordered from companies to produce these new proteins, and tested whether the companies would prohibit them from ordering the dangerous DNA sequences.
The answer? We are not ready at all. The companies have mainly banned the sale of the exact DNA sequences that lead to the production of ricin in its original structure. They have not been able to cope with the improvised sequences, which would have produced improvised ricin.
But it’s not just a ricin problem. Microsoft researchers went on to study 71 other toxins, producing 75,000 variations of them. They used open-source AI engines to do the job—the kind any amateur terrorist could run themselves. And once again, they saw that the companies that are supposed to protect us from the DNA sequences that would produce these variations simply couldn’t do it.
The positive side is that the researchers alerted the relevant companies and the government to this breach.
"We have contacted the relevant agencies regarding this potential vulnerability… including policy contacts at the Department of Science and Technology, the National Institute of Standards and Technology, the Department of Homeland Security, and the Office of Pandemic Preparedness and Response in the United States," They wrote"The details were kept confidential until more comprehensive research could be conducted… to develop and implement software updates."
The updated software is now capable of detecting most of the DNA sequences that will lead to the creation of the new toxins. It still can't handle well about three percent of all new sequences, but that's probably enough to put a significant obstacle in the way of any bioterrorist starting out. It's likely that if that bioterrorist tries to order dozens of variations of toxic proteins, and is rejected time and time and time again, the computer systems will warn that "something strange is happening here," and send a team of FBI agents to his door. At least, one can hope so.
Another important point is that Microsoft researchers did not actually create 75,000 "toxins," but "toxin-like substances." That is, variations that resemble toxins, and some of them are probably at least as harmful as the original toxins, but some are not harmful at all. They have not tested all of these tens of thousands of toxins in the laboratory, so it is difficult to say in advance which one is more or less harmful. Still, it is likely that many of these new toxins are truly dangerous, even more so than the original proteins. This means that even if bioterrorists manage to order the necessary DNA, they will have to experiment with the different versions to finally determine which one is successful enough for their super-attack.
Comforting you? Maybe a little. But it's clear that the danger still exists. That's how it is.
Summary
To me, Microsoft’s research shows that AI can be used as a positive tool: to identify places where our defenses are not yet strong, and to strengthen them in advance. I’m not going to sugarcoat the situation too much. There is going to be an arms race here, as terrorists learn to use these tools to design more malicious threats. But intelligence agencies and governments are not complete fools either, and we can be sure that they are aware of these dangers, and are working to prevent bioterrorists from starting a new global pandemic, or turning Washington or Tel Aviv into a ghost town.
Who will win the new arms race? The one who has the most powerful and creative artificial intelligence, and knows how to use it better. And here's another good reason why the superpowers are currently fighting each other to advance their artificial intelligence capabilities.
Now all that remains is to hope that the good guys will win.
More on the subject on the science website:
