Previous PROJECTS

Projects: Ongoing Previous
Medical Imaging & Data Analysis
Deep learning for efficient MRI
Supervisor(s): Prof. Alex Bronstein, Dr. Michael Zibulevsky, Sanketh Vedula, Ortal Senouf

Magnetic Resonance Imaging (MRI) is a leading modality in medical imaging since it is non-invasive and produces excellent contrast. However, the long acquisition time of MRI currently prohibits its use in many applications – such as cardiac imaging, emergency rooms etc. During the past few years, compressed sensing and deep learning have been in the forefront of MR image reconstruction, leading to great improvement in image quality with reduced scan times. In this project, we will work towards building novel techniques to push the current benchmarks in deep learning based MRI. 

Deep Learning Algorithms & Hardware
How quantization noise affects accuracy?
Supervisor(s): Chaim Baskin

When quantizing a neural network, it is often desired to set different bitwidth for different layers. To that end, we need to derive a method to measure the effect of quantization errors in individual layers on the overall model prediction accuracy. Then, by combining the effect caused by all layers, the optimal bit-width can be decided for each layer. Without such a measure, an exhaustive search for optimal bitwidth on each layer is required, which makes the quantization process less efficient.
The cosine-similarity, mean-square-error (MSE) and signal-to-noise-ratio (SNR) have all been proposed as metrics to measure the sensitivity of DNN layers to quantization. We have shown that the cosine-similarity measure has significant benefits compared to the MSE measure. Yet, there is no theoretical analysis to show how these measures relate to the accuracy of the DNN model.

In this project, we would like to conduct a theoretical and empirical investigation to find out how quantization at the layer domain effects noise in the feature domain. Considering first classification tasks, there should be a minimal noise level that cause miss-classification at the last layer (softmax). This error level can now be propagated backwards to set the tolerance to noise at other lower layers. We might be able to borrow insights and models from communication systems where noise accumulation was extensively studied.

Exploring the expressiveness of quantized neural networks
Supervisor(s): Chaim Baskin

It has been shown that it is possible to significantly quantize both the activations and weights of neural networks when used during propagation, while preserving near-state-of-the-art performance on standard benchmarks. Many efforts are being done to leverage these observations suggesting low precision hardware (Intel, NVIDA, etc). Parallel efforts are also devoted to design efficient models that can run on CPU, or even on the mobile phone. The idea is to use extremely computation efficient architectures (i.e., architectures with much less parameters compared to the traditional architectures) that maintain comparable accuracy while achieving significant speedups.

In this project we would like to study the trade-offs between quantization and over-parameterization of neural networks from a theoretical perspective. At a higher level we would like to study how these efforts for optimizing the number of operations interacts with the parallel efforts of network quantization. Would future models be harder for quantization? Can HW support of non-uniform quantization be helpful here?