​Convolutional neural networks and MRI 

Among the most prominent artificial intelligence approaches, deep learning and its convolutional neural networks (CNN) are of particular interest to developers of magnetic resonance image reconstruction algorithms. Their benefit is twofold: First, artificial neural networks make it possible to recover images of a quality similar to those obtained from complete data sets by collecting only a small fraction of the data, typically one eighth of the total available. Second, these algorithms reconstruct the images much faster than previous methods (10 to 100 times faster). Ultimately, patients reap the benefits as the hospital examination time is tremendously reduced and as such, more people can be scanned on a daily basis.

The fastMRI collaborative project

In the interest of advancing medical technology AI of course attracts the giant digital companies. Facebook AI Research and NYU Langone Health recently launched fastMRI, a collaborative open data research project. Their goal? Assessing the performance of CNNs under development around the world to create a reference benchmark. NYU Langone Health released full raw MRI data sets of the knee in 2018 and of the brain in late 2019. For brain imaging, data was collected from 6,970 individuals, on 11 different Siemens Healthineers scanners, at 1.5 or 3 teslas (43% of the data was collected on 1.5 T MR systems), from five partner sites in the United States. They also mixed several imaging contrasts (T1, T2 or FLAIR weighting) which provided access to additional information on brain tissue. The data are freely accessible (cf. https://fastmri.med.nyu.edu/ for details).

Cosmic: a deep learning project to reduce the scan time and speed up image reconstruction in MRI

The release of open access data prompted Zaccharie Ramzi, a doctoral student under the supervision of Philippe Ciuciu (PARIETAL Inria-CEA team, NeuroSpin, Joliot Institute) and Jean-Luc Starck (Cosmostat, Irfu CEA-CNRS) to establish a comparative analysis of several deep neural networks used in MRI^[1]. This benchmark is a concrete application of their common Cosmic project (Compressed Sensing for Magnetic resonance Imaging and Cosmology) which pools the deep learning skills of the two teams to reduce the scan time and speed up image reconstruction without degrading image quality. Cosmostat brings to the project its expertise on the reconstruction of images from large astrophysical surveys such as algorithms, 3D image reconstructions and their representation. The NeuroSpin team, for its part, brings its knowledge in MRI acquisition and in software engineering. The issues to overcome between astrophysical and MRI images are the same: Missing data and background noise that dominates the measurements. This collaboration has already enabled important advances with astrophysical image calculation times reduced by ten after optimization and acquisition times in 2D multislice imaging dropping from 4 minutes to 18 seconds thanks to the compressed strategies implemented on MR systems at NeuroSpin^[2].

The Brain fastMRI Challenge

The first experience in 2019 on the fastMRI data, solidified the researchers ability to ensure the robustness of pre-existing algorithms and test their own development. When Facebook AI Research and NYU Langone Health announced in Oct 2020,the international “Brain FastMRI Challenge”, aiming at comparing the performance in image reconstruction of different CNN architectures from under-sampled data, corresponding to faster examinations as they could be done in the future, CEA the CEA experts seized the opportunity to work on this challenge.

On October 1st, the competition test data set was released, artificially under-sampled by a factor (R) 4 or 8, to match 4- or 8-fold shorter scan times. Prior to that, the international community had been able to train its CNN architectures for several months on training and validation data sets in a supervised manner, that is, with knowledge of the fully sampled data. The challenge with test data is of course that no ground truth (or complete data) is available, making the competition blind and thus more challenging yet fair to all contenders. In addition to these two 4x and 8x tracks, the challenge also included an unsupervised transfer learning axis, the aim of which was to test neural networks on data from General Electric or Philips MR systems while they had been trained on Siemens Healthineers data only.

The deadline for submitting results was October 15. The jury proceeded in two stages. First, quantitative metrics were calculated, including the similarity score between the reference images, reconstructed from the complete data only available to the organizers, and those reconstructed by the participants with their algorithms (Structural Similarity - SSIM). Then, the images were visually evaluated by radiologists on specific criteria (artifacts, image sharpness, contrast / noise ratio).

The organizers announced the results of the competition on December 11, 2020, at the Neural Information Processing Systems (NeurIPS) international conference, the annual top-level conference in machine learning. Our NeuroSpin/Cosmostat research trio placed second (runner-up) in the two supervised categories out of 19 competitors. A success all the more deserved as the competition was stiff and against companies as well as academia. An article dedicated to this challenge and co-signed by the organizers of the challenge, in particular Matthew Muckley from Facebook AI Research and Florian Knoll, NYU Langone, as well as the three teams of finalists, will be published in 2021 in the journal IEEE Transactions on Medical Imaging.

Zaccharie Ramzi's video at NeurIPS conference

https://slideslive.com/38942421/xpdnet-for-brain-multicoil-mri-reconstruction

[1] Cf. news of 06/04/2020 https://joliot.cea.fr/drf/joliot/Pages/Actualites/Scientistiques/2020/Deep-learning-IRM-Standard-comparaison- reseaux.aspx

[2] Cf. news of 02/17/2019 https://www.cea.fr/drf/Pages/Actualites/En-direct-des-labos/2019/IRM-haute-resolution--comment-ne-pas- spend-hours-.aspx