Compute-In-Memory APU Delivers GPU-Level AI Performance with Lower Energy Usage

GSI Technology, Inc., a pioneer in the development of the Associative Processing Unit (APU), has announced groundbreaking findings regarding its compute-in-memory (CIM) technology. A collaborative research paper from Cornell University reveals that GSI’s Gemini-I APU achieves GPU-level performance while significantly reducing energy consumption. This advancement marks a pivotal moment for artificial intelligence (AI) and high-performance computing (HPC).

Key Findings of the Cornell Study

The research, documented in the paper titled “Characterizing and Optimizing Realistic Workloads on a Commercial Compute-in-SRAM Device,” was presented at the Micro ’25 conference and published by ACM. The study confirmed several critical points:

• GPU-Class Performance: The Gemini-I APU demonstrated throughput comparable to NVIDIA’s A6000 GPU for retrieval-augmented generation (RAG) tasks.
• Energy Efficiency: The APU consumes more than 98% less energy than traditional GPUs across various datasets, highlighting its sustainability.
• Faster Processing: The APU outperforms standard CPUs in retrieval tasks, achieving up to 80% faster processing times.

Implications for the Future

Lee-Lean Shu, the CEO of GSI Technology, emphasized the significance of these findings, suggesting that compute-in-memory could transform the $100 billion AI inference market. The unique memory-centric design of the APU allows for substantial performance gains at a fraction of the energy cost.

Industry Applications

The research underscores the growing demand for energy-efficient computing across various sectors. Potential applications of GSI’s APU include:

• Edge AI for robotics and drones
• IoT devices requiring power conservation
• Defense and aerospace applications with stringent energy demands

GSI Technology also announced plans to advance its product line further. The upcoming Gemini-II APU is anticipated to deliver approximately ten times the throughput of its predecessor while maintaining lower latency for memory-intensive tasks.

Conclusion

The findings from Cornell University enhance the understanding and potential of compute-in-memory technologies in AI and HPC applications. As industries strive for greater energy efficiency, GSI Technology’s innovative solutions may play a crucial role in shaping the future of computing.