Jiby Mariya Jose

Optimizing neural network energy efficiency through low-rank factorisation and pde-driven dense layers

• Authors Details :  
• Jiby Mariya Jose

1K Views Original Article

s deep learning models continue to grow in complexity, the computational and energy demands associated with their training and deployment are becomingincreasingly significant, particularly for convolutional neural networks (CNNs) deployed on CPU-bound and resource- limited devices. Fully connected (FC)layers, while vital, are energy-intensive, accounting for 85.7% of a network’s parameters but contributing only 1% of the computations. This research proposes anovel approach to optimising these layers for greater energy efficiency by integrating low-rank factorisation with differential partial differential equations (PDEs).The introduction of the LowRankDense layer, which combines low-rank matrix factorisation with a differential PDE solver, aims to reduce both the parametercount and energy consumption of FC layers. Experiments conducted on the MNIST, Fashion MNIST, and CIFAR-10 datasets demonstrate the effectiveness ofthis approach, yielding promising results in terms of reduced energy usage and maintaining comparable performance, thereby enhancing the practicality andsustainability of CNNs for widespread use in environments with limited computational resources

Article Subject Details

• Computer Science
• Computer Science Applications
• Neuro imaging
• Neurology
• Neuroscience

Article Keywords Details

• Lightweight CNN Design
• Energy Reduction
• Energy-Efficiency
• CNN
• Computational Efficiency
• Energy-Efficient CNN Design
• Low rank factorization
• Reduce Training Time

Article File

More Article by Jiby Mariya Jose