Improving Performance of Error-Tolerant Applications: A Case Study of Approximations on an Off-the-Shelf Neural Accelerator

Trending workloads and applications are leading many of the new advances in computer architecture and design paradigms. For instance, deep learning applications are transforming the way we do computing. On one hand, specific architectures are currently commercialized as neural processing units, specialized hardware accelerators for these types of applications, achieving significant performance improvements. On the other hand, design paradigms, such as approximate computing, exploit existing inherent tolerance to imprecise computations in these applications to reduce their computation complexity and produce energy-efficient implementations. Relevant available approximations are limited to the software layer to improve the performance of deep learning applications when using an off-the-shelf specialized accelerator alongside edge computing platforms. In this work, we present a case study of performance improvement by introducing approximate computing techniques to three deep learning classification applications. Our test platform is a Raspberry Pi 4, as edge computing device, and a Movidius Myriad X, as neural accelerator. Our experimental results show that using a mixture of approximate techniques can achieve a performance improvement from 20x to 48x with no accuracy degradation for a compute-intensive classification application.