Evaluation of the Robustness of Learned MR Image Reconstruction to Systematic Deviations Between Training and Test Data for the Models from the fastMRI Challenge


    Patricia M. Johnson1, Geunu Jeong2, Kerstin Hammernik3, 4, Jo Schlemper5, Chen Qin3, 6, Jinming Duan3, 7, Daniel Rueckert3, 4, Jingu Lee2, Nicola Pezzotti8, Elwin De Weerdt9, Sahar Yousefi10, Mohamed S. Elmahdy10, Jeroen Hendrikus Franciscus Van Gemert9, Christophe Schülke11, Mariya Doneva11, Tim Nielsen11, Sergey Kastryulin11, Boudewijn P. F. Lelieveldt10, Matthias J. P. Van Osch10, Marius Staring10, Eric Z. Chen12, Puyang Wang13, Xiao Chen12, Terrence Chen12, Vishal M. Patel13, Shanhui Sun12, Hyungseob Shin14, Yohan Jun14, Taejoon Eo14, Sewon Kim14, Taeseong Kim14, Dosik Hwang14, Patrick Putzky15, Dimitrios Karkalousos15, Jonas Teuwen16, Nikita Miriakov16, Bart Bakker8, Matthan Caan17, Max Welling15, Matthew J. Muckley18& Florian Knoll1


    1. Department of Radiology, NYU Langone Health, New York, NY, USA
    2. AIRS Medical, Seoul, Korea
    3. Department of Computing, Imperial College London, London, UK
    4. AI in Healthcare and Medicine, Klinikum Rechts der Isar, Technical University of Munich, Munich, Germany
    5. Hyperfine Research Inc., Guilford, CT, USA
    6. Institute for Digital Communications, School of Engineering, University of Edinburgh, Edinburgh, UK
    7. School of Computer Science, University of Birmingham, Birmingham, UK
    8. Philips Research, Eindhoven, The Netherlands
    9. Philips Healthcare, Best, The Netherlands
    10. Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
    11. Philips Research, 22335, Hamburg, Germany
    12. United Imaging Intelligence, Cambridge, USA
    13. Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
    14. Department of Electrical and Electronic Engineering, Yonsei University, Seoul, South Korea
    15. Amsterdam Machine Learning Lab, University of Amsterdam, Amsterdam, The Netherlands
    16. Radboud University Medical Center, Netherlands Cancer Institute, Amsterdam, The Netherlands
    17. Department of Biomedical Engineering and Physics, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands
    18. Facebook AI Research, New York, NY, USA




    The 2019 fastMRI challenge was an open challenge designed to advance research in the field of machine learning for MR image reconstruction. The goal for the participants was to reconstruct undersampled MRI k-space data. The original challenge left an open question as to how well the reconstruction methods will perform in the setting where there is a systematic difference between training and test data. In this work we tested the generalization performance of the submissions with respect to various perturbations, and despite differences in model architecture and training, all of the methods perform very similarly.