Speeding up Machine Learning Inference on Edge Devices by Improving Memory Access Patterns using Coroutines
Conference Publication ResearchOnline@JCUAbstract
We demonstrate a novel method of speeding up large iterative tasks such as machine learning inference. Our approach is to improve the memory access pattern, taking advantage of coroutines as a programming language feature to minimise the developer effort and reduce code complexity. We evaluate our approach using a comprehensive set of benchmarks run on three hardware platforms (one ARM and two Intel CPUs). The best observed performance boosts were 65% for scanning the nodes in a B+ tree, 34% for support vector machine inference, 12% for image pixel normalisation, and 15.5% for two dimensional convolution. Performance varied with data size, numeric type, and other factors, but overall the method is practical and can lead to significant improvements for edge computing.
Journal
N/A
Publication Name
2022 IEEE 25th International Conference on Computational Science and Engineering (CSE)
Volume
N/A
ISBN/ISSN
979-8-3503-9633-1
Edition
N/A
Issue
N/A
Pages Count
8
Location
Wuhan, China
Publisher
Institute of Electrical and Electronics Engineers
Publisher Url
N/A
Publisher Location
Piscataway, NJ, USA
Publish Date
N/A
Url
N/A
Date
N/A
EISSN
N/A
DOI
10.1109/CSE57773.2022.00011