Speeding up Machine Learning Inference on Edge Devices by Improving Memory Access Patterns using Coroutines

Conference Publication ResearchOnline@JCU
Belson, Bruce;Philippa, Bronson
Abstract

We demonstrate a novel method of speeding up large iterative tasks such as machine learning inference. Our approach is to improve the memory access pattern, taking advantage of coroutines as a programming language feature to minimise the developer effort and reduce code complexity. We evaluate our approach using a comprehensive set of benchmarks run on three hardware platforms (one ARM and two Intel CPUs). The best observed performance boosts were 65% for scanning the nodes in a B+ tree, 34% for support vector machine inference, 12% for image pixel normalisation, and 15.5% for two dimensional convolution. Performance varied with data size, numeric type, and other factors, but overall the method is practical and can lead to significant improvements for edge computing.

Journal

N/A

Publication Name

2022 IEEE 25th International Conference on Computational Science and Engineering (CSE)

Volume

N/A

ISBN/ISSN

979-8-3503-9633-1

Edition

N/A

Issue

N/A

Pages Count

8

Location

Wuhan, China

Publisher

Institute of Electrical and Electronics Engineers

Publisher Url

N/A

Publisher Location

Piscataway, NJ, USA

Publish Date

N/A

Url

N/A

Date

N/A

EISSN

N/A

DOI

10.1109/CSE57773.2022.00011