University of Notre Dame
Browse
- No file added yet -

Statistical Neural Networks: Concepts, Frameworks and Applications

Download (5.87 MB)
thesis
posted on 2020-05-28, 00:00 authored by Tianchen Wang

As one of the fundamental issues in the fields such as computer vision and deep learning, accelerating the inference speed of the convolutional neural network (CNN) has attracted a lot of attention, which is to reduce the gap between the desired high performance of deep CNN and the required huge amount of computation. Various techniques have been explored to address this challenge, such as network compression and quantization, lightweight networks targeting resource-constrained platforms, dynamic computation graphs providing efficient early exits, and, on top of all these techniques, network parallelization. Although a large body of works exists in neural network parallelization, most of them focus on the training phase only or distributing a batch of instances to multiple computation cores. None of these parallelization works, however, helps to reduce the inference latency of a single instance (such as an image), which is critical for real-time applications. Existing techniques for single image inference include operator parallelism and model parallelism. The former technique explores concurrency in operators such as convolution, and the latter distributes kernels of convolutional layers across multiple cores. These approaches do not offer good scalability and usually cannot fully utilize a large number of cores available in modern high-performance computing platforms.

In this dissertation, we introduce a new general framework named statistical convolutional neural network (SCNN) to speedup the instance inference with the help of independent component analysis (ICA), which can be applied to various tasks. As a general framework, SCNN can be implemented with different neural network backbones to speedup, while it is orthogonal to the existing speedup methods. We use ICA to decompose the spatio-temporal correlated data and propagate the extracted essential features as the learning process. The performance evaluations of SCNN in video object detection and 3D cardiac cine MRI segmentation shows that SCNN could achieve a large speedup compared with the existing methods. We further introduce ICA-Net as an improvement of SCNN in accuracy, throughput, and latency. The improved framework is implemented and verified by multiple tasks including image classification, image object detection, video object detection, and 3D cardiac cine MRI segmentation. The latencies of the models are all largely reduced and the accuracies achieve state-of-the-art.

History

Date Modified

2020-07-24

Defense Date

2020-05-01

CIP Code

  • 40.0501

Research Director(s)

Yiyu Shi

Committee Members

Jinjun Xiong Meng Jiang Walter Scheirer

Degree

  • Doctor of Philosophy

Degree Level

  • Doctoral Dissertation

Alternate Identifier

1176465160

Library Record

5719708

OCLC Number

1176465160

Program Name

  • Computer Science and Engineering

Usage metrics

    Dissertations

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC