Semiconductor Engineering

ResNet-50 Does Not Predict Inference Throughput For MegaPixel Neural Network Models

Customers are considering applications for AI inference and want to evaluate multiple inference accelerators. As we discussed last month, TOPS do NOT correlate with inference throughput and you should ...
Semiconductor Engineering

Inference Acceleration: Follow The Memory

Much has been written about the computational complexity of inference acceleration: very large matrix multiplies for fully-connected layers and huge numbers of 3×3 convolutions across megapixel images ...
insideHPC

Groq AI Chip Benchmarks Leading Performance on ResNet-50 Inference

Today AI chip startup Groq announced that their new processor has achieved 21,700 inferences per second (IPS) for ResNet-50 v2 inference. Groq’s level of inference performance exceeds that of other ...
Fudzilla

Tensor Streaming Processor breaks 21,700 inferences per second

Groq, the inventor of the Tensor Streaming Processor (TSP) architecture, today announced that its processor has achieved 21,700 inferences per second (IPS) for ResNet-50 v2 inference. Groq said that ...
insideHPC

Intel Xeon Scalable Processors Set Deep Learning Performance Record on ResNet-50

Today Intel announced a deep learning performance record on image classification workloads. Intel was able to achieve 7878 images per second on ResNet-50 with its latest generation of Intel Xeon ...
Electronic Design

11 Myths About Inference Acceleration

It’s important to understand that an inference accelerator is a completely new kind of chip, with many unknowns for the broader market. In our industry, there’s a learning curve for everything, from ...
The Next Platform

MLPerf Inference Results Offer Glimpse into AI Chip Performance

There is much at stake in the world of datacenter inference and while the market has not yet decided its winners, there are finally some new metrics in the bucket to aid decision-making. Interpreting ...