NDSEG Fellowship for overcoming Moore’s Law with innovative architecture

With the end of Moore’s Law close on the horizon, some leading semiconductor manufacturers have already halted the development of 7nm technology and beyond – a massive shift in the industry roadmap. But demand for computing, as always, marches on undeterred. In tandem with this, improvements to processors have long surpassed that of off-chip memory (DRAM), a phenomenon described as the memory wall. Architects to this point have combatted this with a hierarchy of on-chip memory (SRAM), which has exploded in recent years.

These two trends together have put pressure on architecture researchers for new, innovative designs. With each successive generation of chips, manufacturers see improvements dwindle. Over 2.5 quintillion bytes of data are created every single day, and the number will only continue to grow.

To meet this data processing demand in the post-Moore’s law world, CSE PhD student Charles Eckert is working to expand the role of memory and give it a dual responsibility to both store and compute data. Working with Prof. Reetuparna Das, Eckert was a Department of Defense NDSEG Fellowship for his research, which  focuses on how to accelerate Artificial Intelligence (AI) using in-memory computing.

Today’s architectures are dominated by storage needs with over two-thirds of a processor (its caches) and all main memory devoted to storage. Today these memory elements can only be used as passive storage, but Eckert seeks to answer the natural question: can we use these memory elements to do more than store data?

To answer it, he plans to work on tackling three hypotheses. First, he will design a hardware accelerator built for AI entirely out of memories like SRAM. By merging the memory and computation into one framework, he believes he can produce about 20x higher computation density, as well as save time spent loading data from memory. Second, Eckert will explore similar applications for newer, emerging memory architectures like resistive RAM, magnetic RAM, and flash. And finally, transforming general-purpose code to directions for an architecture he designed previously called Neural Cache. This architecture repurposes cache structures to transform them into massively parallel computational units capable of running inferences for Deep Neural Networks.

“Separating memory and compute is simply a waste of energy and die area,” Eckert says in his proposal. He argues that a hybrid system would instead result in reduction in the size of processors as well as their power consumption, allowing for easier deployment in the field. “Traditional architecture methods are becoming dated and the end of Moore’s law will lead to a significant shift where domain experts and hardware architects work closely to solve computational challenges. It heralds a golden-age for computer architectural innovations.”