The search for a new type of computer / John Poulos

The end of Moore's Law is approaching, and the chip manufacturers are investing billions in the development of new computing technologies

In a tiny, windowless meeting room at the research and development headquarters of Intel, the world's most influential manufacturer of microprocessors and semiconductors, Mark Bohr calmly explains that the well-known Moore's Law is long dead. This may seem surprising given that Bohr, who manages the company's process architecture and integration field, is practically in the Moore's Law business: his job is to find a way to make Intel's transistors, which are 14 nanometers wide, twice as small within ten years. But Bohr doesn't blink behind the round frame of his glasses: "You have to understand that the era of the traditional miniaturization of transistors, where you take the structure and the basic materials and reduce them, ended about ten years ago."
In 1965, Gordon Moore, who was then the director of research and development at Fairchild Semiconductor, published the document with the blunt title: "Squeezing more components into integrated circuits". Moore predicted that the number of transistors that could be put on a chip at an optimal cost would double every year. Ten years later, he updated his prediction, and it became the well-known "Moore's Law": the number of transistors in a computer chip will double every two years.
Integrated circuits are what make a computer work, and the application of Moore's Law is what makes computers evolve. The transistors are the "atoms" of electronic calculation: tiny switches that encode every 0 and 1 in the computer's memory and logic as differences in electrical voltage. Therefore, if you double the number of transistors in a certain physical area, you can double the amount of calculations that can be performed at the same cost. Intel's first general-purpose microprocessor, the 8080, was one of the catalysts for the personal computer revolution when it hit the market in 1974. The silicon rod in it, which was five centimeters long, contained 4,500 transistors. As of this writing, Intel's high-end server central processing units (CPUs), which are the densest chips on the market, contain 4.5 billion transistors per unit. At Intel's manufacturing facilities ("Pab") in Hillsboro, Oregon, the most advanced manufacturing process is able to burn on a silicon substrate lines that are only 14 nanometers in size: narrower than bacterial spores. This exponential increase in transistor density turned the room-sized vacuum tube-powered computing engines of the mid-20th century into the miniaturized silicon wonders of the early 21st century.
But even Moore's Law collapses under the pressure of the laws of physics, and within ten years there will be no possibility to maintain this unprecedented rate of miniaturization. Therefore, chipmakers such as Intel, IBM and Hewlett-Packard (HP) are investing billions in research and development to deal with the post-Moore's Law world. This requires abandoning basic assumptions about how our technology works. Does a computer chip have to be a XNUMXD array of wires baked into silicon? IBM thinks not, and is deeply investigating the possibility of using nanotubes made of carbon and graphene as a substrate for computing. And what about the electrons, are they essential? IBM and HP are also betting on photonics, where short flashes of light replace electrical voltages.
HP goes further than that: the company wants to expand the very basic theory of electronics. The company's engineers built a prototype of a computer, code-named "the machine", which makes use of the "missing link" of electronics: the memory-restore. This component, described mathematically decades ago but only recently developed, enables the combination of the storage and random access memory (RAM) features of the computer. Thanks to memorristors, the common metaphor of the CPU as the "brain" of the computer will be more accurate than for transistors, because memorristors work more like neurons: they transmit and encode information, as well as store it. Such a combination between volatile memory and long-term storage could dramatically increase efficiency and reduce the "von Neumann bottleneck", which limited computing for half a century.
None of these technologies will be able to replace, or even strengthen, the chips in our laptops and smartphones in the coming years. However, by the end of the decade, at least one of them must reach a level of computational performance that will allow it to replace traditional silicon circuit engineering, when it reaches its inevitable end. The question is which of the technologies, and when.

beyond silicon

The idea of ​​Moore's Law is simple: halving the transistors means twice the computational performance at the same cost. In practice, the story was always more complex. Moore's 1965 paper may have predicted what would happen to transistor density every two years, but he did not explain how this increased density would result in a doubling of performance. It took another nine years until Robert Dennard, a scientist at IBM, published an explanation known today as the "Dennard Scale". In this explanation he described how the power density of MOSFETs, the dominant technology in 1974, remained constant even as they got smaller. In other words, shrinking the transistors means that the electrical voltage and current needed to switch them also decreases.

For thirty years, Dennard's scale served as the secret driving force behind Moore's Law, ensuring continuous improvements in computer performance that helped people start businesses, design products, cure diseases, navigate spacecraft, and democratize the web. Then the scale stopped working. As soon as the production facilities began to burn in silicon signals smaller than 65 nanometers (about half the length of the HIV virus), the chip designers discovered that electrons began to "leak" from their transistors due to the effects of quantum mechanics. Devices have become too small to reliably switch from "on" to "off", and a digital computer that cannot differentiate between 0 and 1 is in serious trouble. Researchers at IBM and Intel also discovered a "frequency barrier," which limits the speed at which a silicon-based CPU can perform logic operations, roughly four billion operations per second, without overheating.

Technically, Moore's Law could continue (and it did): Intel kept cramming smaller and smaller transistors into its silicon substrates every two years, but that didn't translate directly into cheaper and faster computers.

Since 2000, chip engineers faced with these obstacles have developed sophisticated workarounds. They got around the frequency barrier by creating multi-core chips (a 10 GHz processor will burn itself out, but four, eight, or sixteen 3 GHz processors working together will not be harmed). They blocked the leaking transistors with "triple gates" that control the electric current from three sides instead of just one. They've also built systems that allow processors to outsource particularly difficult tasks to dedicated assistants (the iPhone 6's screen, for example, is managed by a dedicated quad-core graphics processor). But all these patches do not change the fact that the era of shrinking silicon devices is less than a decade away.

This is why some chip manufacturers are looking for ways to get rid of silicon. In 2014, IBM announced that it was allocating three billion dollars to feverish research into various forms of computing beyond silicon. The material at the center of the investigation is graphene: sheets of carbon that are only one atom thick. Like silicon, graphene has properties useful for electronics that remain stable over a wide range of temperatures. Another advantage is that the electrons pass through it at relative speeds, and the most important advantage is that graphene devices also deal with changes in scale, at least in the laboratory. Transistors have already been built from graphene that are capable of operating at speeds hundreds and thousands of times greater than the leading silicon devices, with reasonable power density, even below the five nanometer threshold where silicon succumbs to quantum effects.

What graphene lacks compared to silicon is the "band gap", an energy difference between the orbitals in which the electrons are bound to the atom, and those in which the electrons are free to move and cause electrical conductivity. In metals, for example, there is no such gap: they are "pure" conductors. Without such an energy gap, it is very difficult to stop the flow and switch the transistor from on to off, so the graphene transistor cannot reliably encode digital logic. "We were the leaders in this field," admits Supertik Guha, director of physical sciences at IBM's Thomas J. Watson Research Center, "but the results we obtained with graphene were not encouraging. It must be very cheap and also offer a unique advantage to take the place of existing materials. It has very interesting features, but we were unable to identify the winning application in it.”

Carbon nanotubes may be more successful. When graphene sheets are rolled into hollow cylinders, a small band gap is created in them, which gives them semiconductor properties similar to those of silicon. This reopens the possibility of using them to create digital transistors. "We are optimistic and cautious," says Guha. "As single devices, carbon nanotubes at the scale of ten nanometers perform better than anything else available. Based on our simulations of computing systems using carbon nanotubes, we can expect a fivefold improvement [compared to silicon] in performance or energy efficiency.”
But carbon nanotubes are delicate structures. If the diameter of the tube, or its chirality (the angle at which the atoms are "rolled"), changes even slightly, the gap between the lines may disappear and it will become useless as a digital circuit component. The engineers will also have to find a way to place billions of nanotubes in neat rows and a few nanometers apart, using the same technology that silicon manufacturing facilities rely on today. "In order for the carbon nanotubes to be worthy successors of silicon, we need to solve all of this within two to three years," Guha says.

Break the memory barrier

"What is the most expensive real estate on earth?" asks Andrew Wheeler. "Here, here." He points to a box drawn with a black marker on an eraser board, which represents the piece of silicon inside a chip. Wheeler, a tall, thin, square-jawed man in jeans and a plaid cotton shirt, looks more like an ex-cowboy than the VP of HP Laboratories, the research arm of Hewlett-Packard. He explains that in practice most of the transistors that occupy this precious space are not used for computing, but for "cache memory", or static random access memory (SRAM), whose entire function is to store instructions that are accessed frequently. This is the silicon equivalent of the PC desktop: the place where you put things you don't want to look for again and again. Wheeler wants this memory to disappear - but it is sooner rather than later, and at this point he will be satisfied with getting rid of the hard drive and the computer's main memory.
According to HP, these three items, collectively known as the "memory hierarchy" with SRAM at the top and hard drives at the base, are responsible for most of the problems encountered by engineers dealing with Moore's Law. Faster processors are useless without large capacity fast memory to store the bits and transfer them as fast as possible.
To break this "memory barrier", Wheeler's team in Palo Alto, California designed a new type of computer, "the machine", which tries to escape the limitations of the memory hierarchy by consolidating it into a single layer. In the memory hierarchy, each layer excels at certain things and fails at others. SRAM is extremely fast, so it can keep up with the CPU, but it's energy-hungry and low-capacity. The main memory, dynamic random access memory (DRAM), is quite fast, dense and durable, which is good because it is the substrate on which the computer runs the active applications. But a power outage causes all its contents to disappear, so "non-volatile" storage media, such as flash drives and hard drives, are required for long-term data storage. These have high capacity and low power consumption, but are terribly slow, and flash memory wears out quite quickly. Because the advantages and disadvantages of the different types of memory complement each other, modern computers combine them so that processors can move information up and down the hierarchy as efficiently as possible. "It's an engineering marvel," says Wheeler, "but also a colossal waste."

Wheeler says universal memory, which would combine the speed of SRAM, the durability of DRAM and the capacity and energy efficiency of Flash, has been considered the holy grail of engineers, designers and programmers for decades. The "machine" makes use of an electronic component called a memristor, to meet the last two requirements. The name of this component is an English abbreviation of "resistor with memory", due to its ability to conduct electricity according to the amount of current that passed through it before. Mathematically, it was predicted in 1971, and for a long time it was thought impossible to build it. But in 2008 HP announced that it had succeeded in building a working memory restorer. The research program was put on the fast track and became the basis for the "machine".
A brief exposure of a memory cell of the memristor type to an electrical voltage can change its conductivity state, thus creating the clear separation between "on" and "off" necessary for storing digital information. Similar to what happens in flash memory, the state of the component is preserved even when the current stops, and similar to DRAM memory, the cells can be very crowded, and read from and written to at high speed.

However, to achieve SRAM-level performance, the memory resistor cells need to be placed next to the main processor, on the same piece of silicon, which is a physically impractical arrangement in today's technology. Instead, HP plans to connect the high-speed memristor memory with the SRAM cache memories found in the logic chips using photonics: transferring bits as flashes of laser light instead of an electric current. It's not exactly the holy grail of universal memory, because the memory hierarchy in the "machine" shrinks from three layers to just two, but it comes close.

Computers with an architecture that will be based on memorristors that combine RAM with non-volatile storage, such as "the machine", will be able to bring great improvements in computer performance without relying on miniaturization along the lines of Moore's Law. IBM's Watson supercomputer, the version that beat the human contestants on Jeopardy in 2011, needed 16 terabytes of DRAM, stored in ten power-hungry Linux server cabinets, to perform its task in real time. The same amount of non-volatile flash memory can fit into a shoebox, and need the same amount of power as a typical laptop. A memory architecture that combines the two functions will make it possible to hold in active memory huge amounts of data for real-time processing, instead of chopping them into small serial blocks, and at much lower energy costs.

According to Wheeler, as more and more devices are connected to the "Internet of Things", the future of Moore's Law is even more in doubt, because of the problem of streaming countless petabytes of information back and forth to data centers for storage and processing. If universal memory manages to squeeze supercomputer capabilities into smaller, more cost-effective packages, we could store and pre-process that data locally, on the connected devices themselves. If the memory barrier is broken, we will no longer care if silicon processors are never smaller than seven nanometers or faster than four gigahertz.

After von Neumann

Even if HP succeeds in its move to create universal memory, computers will remain what they have been since ENIAC, the first general-purpose computer, built in 1946: very fast calculators. Their basic design, formally formulated by the mathematician John von Neumann in 1945, includes a processing unit for executing instructions, a memory pool for storing those instructions and the data on which they are executed, and a connection ("channel") linking the two. The von Neumann architecture is optimally adapted to the linear execution of symbolic instructions, what we call "arithmetic operations". The first computers were actually humans who were paid to sit in a room and perform manual calculations, and it is no coincidence that they designed the electronic computers to automatically and with fewer errors perform the same exhausting process.

Today, we increasingly need computers to perform tasks that cannot be translated well into linear mathematical instructions: tasks such as identifying certain objects in videos that last many hours, or guiding autonomous robots in changing or dangerous areas. Such tasks have more in common with the sensing and pattern-matching abilities of biological brains than with mechanical calculators. Organisms need to extract from the environment in real time information that will allow decision-making. If a fly had to pass discrete instructions back and forth, one after the other, between separate memory and processing modules in its brain, it would not be able to complete the calculation in time to escape the rolled-up newspaper.

Dharmendra Moda, founder of the cognitive computing group at IBM, wants to build computer chips that will be at least as "smart" as the housefly, and just as energy efficient. The key, he says, is to abandon the calculator-like von Neumann architecture. Instead, his team is trying to eventually mimic the columns in the mammalian cerebral cortex, which process, transmit and store information in the same structure, without channels that create bottlenecks. IBM recently unveiled the TrueNorth chip, which contains more than five billion transistors, arranged in 4,096 neurosynaptic cores simulating one million neurons and 256 million synaptic connections.

This arrangement manages to match patterns in real time, with the energy budget of a laser pointer. Moda points to a screen in the corner of the demo room at IBM's Alameda research campus in San Jose, California. The screen shows an image that looks like a security shot from a camera that needs urgent repair: cars, pedestrians and bicycles freeze in place in a traffic circle. One of the pedestrians is marked in a red frame drawn on top of the display. A minute later, the cars, people and bikes jump to another frozen position, as if the video had jumped forward.
"You see, these are not static images," Moda explains. "This is a video from the Stanford campus, analyzed on a laptop simulating the TrueNorth chip. It just runs a thousand times slower than the chip." The actual TrueNorth chip, which normally runs for the video analysis, was being used at the time in an indoor tutorial in the adjacent auditorium, so I couldn't see it in action. Otherwise, says Moda, the video would be displayed in real time, and the small red frames would seamlessly follow the pedestrians from the moment they entered the image frame to the moment they exited it.

Like their counterparts in the von Neumann architecture, neurosynaptic devices such as TrueNorth have inherent weaknesses. "You wouldn't want to run the iOS operating system on that chip," says Moda. "I mean, it's possible, but it would be very inefficient, just as the computer here doesn't efficiently process the video." IBM intends to harness the capabilities of the two architectures: one for precise logical calculations and the other for rapid associative pattern matching, to create a holistic computing system.

In this vision, the classic formulation of Moore's Law still has meaning. Moda's team has already connected 16 TrueNorth chips to one board, and by the end of 2015 they intend to stack eight boards in one device, the size of a cell phone and with a power consumption of one hundred watts, with computing capabilities that "to simulate them in a simulation would require an entire computer center."

In other words, the silicon and the transistor count still matter, but what's even more important is how they're arranged. "That was our insight," says Moda. "With the help of rearranging the building blocks, the building itself takes on a completely different functionality. Many people, including us at the beginning, believed that the technology needed to be changed in order to gain an advantage. In practice, it became clear that even if a new technology can bring advantages, a different architecture can bring about a huge improvement in performance, at a much lower cost."

Moore's Laws

Meanwhile, in the RA3 building in Hillsborough, Michael Mayberry, director of component research at Intel, dispels another myth about Moore's Law: He never really worked with transistors. "The name of the game is cost per function," he says. Whether measuring transistors per square centimeter of silicon, code instructions executed per second, or performance per watt, what matters is getting more work done with fewer resources. It is not surprising that on Intel's website, Moore's Law is not presented as a technological trend or a law of nature, but as a business model.

"When someone asks me 'What about Moore's Law makes you sleepless?' I say, 'I'm a great sleeper,'" says Mayberry. "We didn't stop when Dennard's scale came to an end, we just changed. If you look 15 years ahead, you can see some changes that will take place, but that doesn't mean we will stop." What Intel, IBM and HP agree on is that the future of computing performance, that is, how the industry will deliver improved performance at reduced cost, will no longer look like a line or a curve, but more like the multi-branched tree of biological evolution itself.

The reason for this is the evolution of our vision of computers themselves. It turns out that we don't want independent and omniscient thinking machines, as estimated by many science fiction writers at the end of the 20th century. It is not Moore's Law that is coming to an end, but the era of efficient general computing that Moore's Law described and is possible, or as Mayberry puts it, "cramming everything possible into a box."

Instead, the factor that will drive the relentless race to reduce cost per function will be heterogeneous computing, and Moore's Law will split into several Moore's Laws. Companies such as Intel, IBM, HP and others will integrate not only circuits, but entire systems that will be able to meet the growing demands of separate computing loads. Bernard S. Myerson of IBM says that people buy functions, not computer chips, and in fact are less and less interested in buying computers at all. We simply want our tools to perform calculations, or "think", in ways that make them useful in the contexts in which we work with them. Instead of the super-intelligent computer HAL from the movie 2001: A Space Odyssey, we have Google Now on our smartphone that tells us when to leave for the airport to make our flight on time.
Futurists such as Nick Bostrom (author of the book "Superintelligence: Trajectories, Dangers and Approaches") assume that Moore's Law will cause general artificial intelligence to develop and crystallize into a kind of all-knowing and all-powerful digital entity. Heterogeneous computing, on the other hand, dictates that computing will permeate outward into systems, niches and objects that were previously "stupid". Things like cars, network routers, medical diagnostic equipment, and retail supply chains will gain semi-autonomous flexibility and context-specific capabilities, at the level of domestic animals. In other words, in the world after Moore's Law, computers will not become gods, but will function as a kind of very smart dogs.
And just as a Great Dane is not built for the duties of a terrier, so a graphics processor is not built to do the work of a main processor. HP's Wheeler gives the example of multiple dedicated processing cores "bolted" into a pool of universal memory on the order of petabytes: a combination of processing power and massive memory, which works similarly to the way graphics accelerators and cache memories are arranged around centralized CPU resources today.

At IBM, Moda envisions golf-ball-sized devices with cognitive chips attached to inexpensive cameras that could be dropped into areas affected by natural disasters and identify very specific patterns, such as injured children. Computer scientist Leon Chua of the University of California, Berkeley, who first proposed the idea of ​​memorristors in 1971, says that HP's attempts to collapse the memory hierarchy and IBM's research into re-inventing the mainframe are complementary responses to what he calls "the great bottleneck of The data.” According to him, "It is amazing, that the computers that have been used for everything in the last forty years are still based on the same idea" of von Neumann's calculator-like architecture. The transition on two fronts to heterogeneous computing is "inevitable," says Chua, and "will create a completely new economy", partly because computing after Moore's Law and von Neumann will require completely new methods of programming and system design. According to him, a very large part of computer science, engineering and chip design deals with disguising the built-in limitations of the memory hierarchy and von Neumann's architecture on computing, so as soon as these limitations are removed, "every computer programmer will have to go back to school."
What Chua, Wheeler and Moda don't mention in their predictions for the near future is transistors, or the performance improvements the world has come to expect from them. According to IBM's Myerson, what Moore's Law has accurately described for half a century, a clear relationship between increasing transistor density and decreasing cost and function, may actually be just a temporary coincidence. "If we look at the last forty years in semiconductors, we see a very regular pulse," Myerson says. "Progress will not stop, but technology has developed a heart rhythm disorder."

About the author

John Pavlus is a writer and director who focuses on science, technology and design. His works have been published, among others, in Wired, Nature and MIT Technology Review.

in brief

Progress in computing depends on Moore's Law: the idea that every two years the number of transistors in computer chips doubles. But the laws of physics put a limit to this reduction, and soon the engineers will reach this limit.

Therefore, chip manufacturers are investing billions of dollars in developing completely new computing architectures and chip designs, some of which are based on new materials. Ideas that have been studied for a long time in laboratories are now being tested in industry in full force.

It is too early to tell which technology will win. The most likely outcome is that certain technologies will be used to perform certain tasks that were previously assigned to a single central processing unit. Moore's law, singular, will be replaced by several Moore's laws.

More on the subject

Cramming More Components onto Integrated Circuits. Gordon E. Moore in Electronics, Vol. 38, no. 8, pages 114-117; April 19, 1965.
Memristor: The Missing Circuit Element. LO Chua in IEEE Transactions on Circuit Theory, Vol. 18, no. 5, pages 507-519; September 1971.
Design of lon-Implanted MOSFET's with Very Small Physical Dimensions. RH Dennard et al. in IEEE Journal of Solid-State Circuits, Vol. 9, no. 5, pages 256-268; October 1974.
Carbon Nanotubes: The Route toward Applications. Ray H. Baughman et al. in Science, Vol. 297, pages 787-792; August 2, 2002.

Superintelligence: Paths, Dangers, Strategies, Nick Bostrom, Oxford University Press, 2014

The Next Twenty Years of Microchips, Scientific American Israel, April-May 2010 issue, page 48;

Simply add a memory, Massimiliano Di Ventra and Yuri V. Pershin, Scientific American Israel, June-July 2015 issue, page 50;

The article was published with the permission of Scientific American Israel