Home    Details    Workshop    Register    Download    Submit    Results    Organizers 

Results

When teams submit their segmentation results, the evaluation results will be sent to the team contact person by e-mail and will be listed below.

Rank Team name Submission name Date Sum Scores Sequences used Speed Doc
1TailHot Multi-modality aggregation network3 13‑04‑18 38 T1; T1_IR; FLAIR ~13 sec doc
2WTA2 3D Cascade convolutional architecture - Method 22 23‑05‑18 46 T1; T1_IR; FLAIR ~2 min doc
3LIVIA_ETS HyperDenseNet2 06‑02‑18 63 T1; T1_IR; FLAIR ~4 min doc
4CU_DL2 3D Deep Learning; voxnet2 28‑06‑16 73 T1; T1_IR; FLAIR ~2 min doc
5CU_DL 3D Deep Learning; voxnet13 16‑06‑16 76 T1; T1_IR; FLAIR ~2 min doc
6MSL-SKKU Deep Convolutional Neural Network 19‑06‑17 81 T1; T1-IR; FLAIR ~1.5 min doc
7LRDE Fully Convolutional Network 20‑12‑16 81 T1 ~2 sec doc
8MDGRU Multi-Dimensional Gated Recurrent Units3 27‑07‑16 104 T1; T1_IR; FLAIR ~2 min doc
9FBI/LMB Freiburg U-Net (3D) 01‑05‑16 108 T1-1mm; T1-IR; FLAIR ~2 min doc
10PyraMiD-LSTM2 NOCC with rounds3 23‑05‑16 110 T1; T1-IR; FLAIR ~2 min doc
11AOC Atlas of Classifiers 24‑12‑17 120 T1 ~6 sec doc
12IDSIA PyraMiD-LSTM 05‑06‑15 127 T1; T1_IR; FLAIR ~2 min doc
13STH Hybrid ANN-based Auto-context method2 03‑06‑16 141 T1; T1-IR; FLAIR ~ 5 min doc
14ISI-Neonatology Multi-stage voxel classification 31‑05‑14 145 T1 ~1.5 hours doc
15UNC-IDEA LINKS:Learning-based multi-source integration 09‑02‑15 150 T1; T1_IR; FLAIR ~3 min doc
16BCH_CRL_IMAGINE 3D patch-wise DenseNet and Patch Fusion2 24‑05‑18 171 T1; T1-IR; FLAIR ~2 min doc
17MNAB2 Random Forests 21‑02‑14 174 T1; T1_IR; FLAIR ~25 min doc
18KSOM GHMF ASeTs: MAP-Based with Manifold learning 13‑05‑14 175 T1; T1_IR; FLAIR ~23 min doc
19WTA 3D Cascade convolutional architecture - Method 13 15‑05‑18 194 T1; T1_IR; FLAIR ~5 min doc
20vicorob UdG T1_F MSSEG using T1 + FLAIR (T1-IR skull) 14‑01‑16 197 T1; IR; FLAIR ~2 min doc
21VBM12 VBM12_r738 with WMHC=2 07‑10‑15 199 T1 ~6 min doc
22BIGR2 Multi-Feature SVM Classification 26‑09‑13 199 T1; T1_IR; FLAIR ~35 min doc
23vicorob UdG T1 MSSEG using only T1 (T1-IR skull) 21‑01‑16 214 T1; IR ~2 min doc
24UofL BioImaging MAP-Based Framework 26‑09‑13 222 T1 ~6 sec doc
25CMIV Model-guided Level Sets and Skeletons 26‑09‑13 243 T1; FLAIR ~3 min doc
26UB VPML Med Multi-Atlas with Multiway Cut 26‑09‑13 248 T1; T1_IR; FLAIR ~30 min doc
27Narsil Segmentation using Ensemble Trees 26‑09‑13 256 T1; FLAIR < 2 min doc
28VIBOT10 M-T1+F Convolutional Neural Network 27‑02‑17 259 T1; FLAIR ~3 min doc
29CVSS Multi-atlas segmentation based on supervoxels 14‑02‑17 262 T1; T1_IR; FLAIR ~2 min doc
30drai VGG based Fully Convolutional Neural Network 20‑01‑17 267 T1 ~2 sec doc
31bigr_neuro Automatically Trained kNN Classifier 26‑09‑13 267 T1; FLAIR ~2 hours doc
32S2_QM2 Atlas-based segmentation and AdaBoost 28‑07‑14 268 T1; T1-IR; FLAIR ~45 min doc
33KM-MRF-MAS MAP on Intensity, and Local and Multi-atlas Priors 28‑10‑16 275 T1 ~10 min doc
34Robarts Multi-Atlas with Hierarchical Max-Flow 26‑09‑13 277 T1-1mm; T1-IR ~16 min doc
35SPM12_T1_F* SPM12 with T1 and FLAIR sequence 31‑08‑15 282 T1; FLAIR ~3 min doc
36SPM12_T1_IR* SPM12 with T1 and IR sequence 31‑08‑15 298 T1; IR ~3 min doc
37SJCE Fuzzy Clustering 12‑04‑17 299 T1 <1 min doc
38SPM12_T1* SPM12 with T1 sequence 31‑08‑15 313 T1 ~3 min doc
39MNAB Random Decision Forests 26‑09‑13 313 T1; T1-IR; FLAIR ~15 min doc
40SPM12_T1_IR_F* SPM12 with T1; IR and FLAIR sequence 31‑08‑15 345 T1; IR; FLAIR ~4 min doc
41FSL-Seg* FSL (v5.0) seg file 31‑08‑15 347 T1 ~10 min doc
42LUH-TNT Level-sets with dictionary learning 27‑10‑14 356 T1 < 1 min doc
43FreeSurfer* FreeSurfer (v5.3.0) 31‑08‑15 358 T1-1mm ~1 hour doc
44FSL-PVSeg* FSL (v5.0) PVSeg file 31‑08‑15 360 T1 ~10 min doc
45Jedi Mind Meld Automated Walks using Machine Learning 26‑09‑13 372 T1; T1-IR; FLAIR ~27 sec doc
46mlmi2014ss AdaBoostM2 28‑07‑14 379 T1; T1-IR; FLAIR ~5 min doc
47S2_QM Bayesian-based Adaptive Mean-Shift 26‑09‑13 389 T1; T1-IR; FLAIR ~1.5 hours doc
48CSIM-Lab Multiple Logistic Classification 24‑10‑17 401 T1 doc
49ISTB UniBe Decision Forests with spatial regularization 04‑06‑14 417 T1; T1-IR; FLAIR ~3 min doc
50LNMBrains Gaussian Intensity Model 26‑09‑13 417 T1; T1-IR; FLAIR ~5 min doc
51ECOESH Modified Expectation Maximization 06‑01‑14 458 T1 ~5 min doc
*These methods (SPM, FSL and FreeSurfer) were applied by the MRBrainS organizers as described in the MRBrainS journal paper.
2,3At the end of the submission name indicates that this is the 2nd or 3rd submission (max 3 resubmissions allowed).

Note: As described below, when teams have equal scores, the standard deviations of the three tissue types and the three evaluation measures over all patients are taken into account.

How is the ranking determined?

Ranking is based on the evaluation results of all 15 test datasets and determined as follows: For each evaluation measure (Dice Coefficient (DC), Modified Hausdorff Distance (HD) and Absolute Volume Difference(AVD)), the mean value over all 15 datasets is determined for white matter (WM), gray matter (GM) and cerebrospinal fluid (CSF). Each team receives a rank (1=best) for each tissue type (GM, WM, CSF) and each evaluation measure (DC, HD-95, AVD) based on the mean value of the evaluation measures over all 15 datasets. The final score is determined by adding the ranks of all tissue types and evaluation measures for each team. The team with the lowest score is ranked number 1. In case two teams would have an equal score, the team with the lowest standard deviation over the tissue types will be ranked first.

Per tissue rankings and per measure rankings are presented below.

Overall Segmentation Results

GM Score WM Score CSF Score Rank Sum
1 TailHot 10 1 WTA2 4 1 TailHot 6 1 TailHot 38
2 WTA2 15 2 LIVIA_ETS 16 2 IDSIA 15 2 WTA2 46
3 LIVIA_ETS 21 3 LRDE 16 3 MDGRU 19 3 LIVIA_ETS 63
4 LRDE 24 4 CU_DL 19 4 CU_DL2 21 4 CU_DL2 73
5 CU_DL2 30 5 CU_DL2 22 5 MSL-SKKU 22 5 CU_DL 76
6 CU_DL 30 6 TailHot 22 6 LIVIA_ETS 26 6 MSL-SKKU 81
7 MDGRU 34 7 MSL-SKKU 25 7 CU_DL 27 7 LRDE 81
8 MSL-SKKU 34 8 AOC 32 8 WTA2 27 8 MDGRU 104
9 WTA 40 9 PyraMiD-LSTM2 36 9 STH 27 9 FBI/LMB Freiburg 108
10 FBI/LMB Freiburg 42 10 FBI/LMB Freiburg 37 10 PyraMiD-LSTM2 29 10 PyraMiD-LSTM2 110
11 KSOM GHMF 45 11 WTA 37 11 FBI/LMB Freiburg 29 11 AOC 120
12 PyraMiD-LSTM2 45 12 UNC-IDEA 40 12 MNAB2 40 12 IDSIA 127
13 STH 45 13 ISI-Neonatology 46 13 AOC 41 13 STH 141
14 ISI-Neonatology 45 14 BIGR2 50 14 LRDE 41 14 ISI-Neonatology 145
15 AOC 47 15 MDGRU 51 15 BCH_CRL_IMAGINE 48 15 UNC-IDEA 150
16 BCH_CRL_IMAGINE 50 16 VBM12 56 16 vicorob UdG T1 49 16 BCH_CRL_IMAGINE 171
17 BIGR2 51 17 IDSIA 57 17 UNC-IDEA 49 17 MNAB2 174
18 Narsil 54 18 vicorob UdG T1_F 64 18 vicorob UdG T1_F 52 18 KSOM GHMF 175
19 IDSIA 55 19 KSOM GHMF 66 19 ISI-Neonatology 54 19 WTA 194
20 UB VPML Med 59 20 UB VPML Med 68 20 UofL BioImaging 61 20 vicorob UdG T1_F 197
21 UNC-IDEA 61 21 MNAB2 69 21 VBM12 63 21 VBM12 199
22 MNAB2 65 22 STH 69 22 KSOM GHMF 64 22 BIGR2 199
23 VIBOT10 M-T1+F 65 23 SPM12_T1_F* 71 23 S2_QM2 73 23 vicorob UdG T1 214
24 drai 78 24 BCH_CRL_IMAGINE 73 24 Robarts 77 24 UofL BioImaging 222
25 UofL BioImaging 79 25 Narsil 74 25 CMIV 80 25 CMIV 243
26 SJCE 79 26 vicorob UdG T1 76 26 VIBOT10 M-T1+F 80 26 UB VPML Med 248
27 CVSS 80 27 CVSS 77 27 drai 81 27 Narsil 256
28 VBM12 80 28 CMIV 79 28 KM-MRF-MAS 83 28 VIBOT10 M-T1+F 259
29 vicorob UdG T1_F 81 29 UofL BioImaging 82 29 SPM12_T1_IR* 87 29 CVSS 262
30 CMIV 84 30 SPM12_T1* 83 30 bigr_neuro 91 30 drai 267
31 bigr_neuro 84 31 MNAB 83 31 BIGR2 98 31 bigr_neuro 267
32 S2_QM2 86 32 bigr_neuro 92 32 SPM12_T1_IR_F* 99 32 S2_QM2 268
33 vicorob UdG T1 89 33 SJCE 96 33 FSL-PVSeg* 100 33 KM-MRF-MAS 275
34 KM-MRF-MAS 94 34 Robarts 98 34 FSL-Seg* 100 34 Robarts 277
35 MNAB 95 35 KM-MRF-MAS 98 35 SPM12_T1_F* 100 35 SPM12_T1_F* 282
36 SPM12_T1* 101 36 SPM12_T1_IR* 102 36 CVSS 105 36 SPM12_T1_IR* 298
37 Robarts 102 37 FreeSurfer* 103 37 Jedi Mind Meld 107 37 SJCE 299
38 SPM12_T1_IR* 109 38 drai 108 38 S2_QM 114 38 SPM12_T1* 313
39 SPM12_T1_F* 111 39 S2_QM2 109 39 WTA 117 39 MNAB 313
40 FreeSurfer* 116 40 LUH-TNT 111 40 UB VPML Med 121 40 SPM12_T1_IR_F* 345
41 LUH-TNT 117 41 VIBOT10 M-T1+F 114 41 mlmi2014ss 123 41 FSL-Seg* 347
42 SPM12_T1_IR_F* 121 42 FSL-Seg* 121 42 SJCE 124 42 LUH-TNT 356
43 FSL-PVSeg* 125 43 mlmi2014ss 121 43 CSIM-Lab 127 43 FreeSurfer* 358
44 FSL-Seg* 126 44 SPM12_T1_IR_F* 125 44 Narsil 128 44 FSL-PVSeg* 360
45 Jedi Mind Meld 126 45 CSIM-Lab 130 45 LUH-TNT 128 45 Jedi Mind Meld 372
46 S2_QM 134 46 ISTB UniBe 135 46 SPM12_T1* 129 46 mlmi2014ss 379
47 mlmi2014ss 135 47 FSL-PVSeg* 135 47 LNMBrains 131 47 S2_QM 389
48 LNMBrains 139 48 Jedi Mind Meld 139 48 MNAB 135 48 CSIM-Lab 401
49 ISTB UniBe 143 49 S2_QM 141 49 FreeSurfer* 139 49 ISTB UniBe 417
50 CSIM-Lab 144 50 LNMBrains 147 50 ISTB UniBe 139 50 LNMBrains 417
51 ECOESH 153 51 ECOESH 153 51 ECOESH 152 51 ECOESH 458

GM = Gray matter, WM = White matter, CSF = Cerebrospinal fluid
Sum = Score GM + Score WM + Score CSF
Score = Rank DC + Rank HD + Rank AVD

Gray Matter Segmentation Results

Rank DC Rank HD Rank AVD GM Score
1 TailHot 86.40 1 LIVIA_ETS 1.34 1 SJCE 5.14 1 TailHot 10
2 LIVIA_ETS 86.33 2 TailHot 1.38 2 drai 5.29 2 WTA2 15
3 CU_DL2 86.15 3 CU_DL2 1.44 3 WTA2 5.43 3 LIVIA_ETS 21
4 CU_DL 86.12 4 LRDE 1.44 4 KSOM GHMF 5.44 4 LRDE 24
5 MSL-SKKU 86.06 5 WTA2 1.45 5 Narsil 5.52 5 CU_DL2 30
6 LRDE 86.03 6 CU_DL 1.47 6 BCH_CRL_IMAGINE 5.70 6 CU_DL 30
7 WTA2 86.00 7 MSL-SKKU 1.52 7 TailHot 5.72 7 MDGRU 34
8 ISI-Neonatology 85.77 8 MDGRU 1.54 8 AOC 5.81 8 MSL-SKKU 34
9 WTA 85.70 9 FBI/LMB Freiburg 1.58 9 UB VPML Med 5.87 9 WTA 40
10 FBI/LMB Freiburg 85.44 10 WTA 1.61 10 bigr_neuro 5.90 10 FBI/LMB Freiburg 42
11 MDGRU 85.40 11 UNC-IDEA 1.62 11 VBM12 5.91 11 KSOM GHMF 45
12 VIBOT10 M-T1+F 85.07 12 ISI-Neonatology 1.62 12 vicorob UdG T1_F 5.98 12 PyraMiD-LSTM2 45
13 PyraMiD-LSTM2 84.89 13 PyraMiD-LSTM2 1.67 13 STH 6.02 13 STH 45
14 IDSIA 84.82 14 IDSIA 1.70 14 LRDE 6.05 14 ISI-Neonatology 45
15 STH 84.77 15 MNAB2 1.70 15 MDGRU 6.09 15 AOC 47
16 BIGR2 84.65 16 UofL BioImaging 1.70 16 BIGR2 6.13 16 BCH_CRL_IMAGINE 50
17 AOC 84.59 17 STH 1.71 17 vicorob UdG T1 6.15 17 BIGR2 51
18 MNAB2 84.50 18 VIBOT10 M-T1+F 1.87 18 LIVIA_ETS 6.19 18 Narsil 54
19 UNC-IDEA 84.36 19 BIGR2 1.89 19 PyraMiD-LSTM2 6.35 19 IDSIA 55
20 BCH_CRL_IMAGINE 84.27 20 KSOM GHMF 1.92 20 CU_DL 6.42 20 UB VPML Med 59
21 KSOM GHMF 84.13 21 Robarts 1.95 21 WTA 6.56 21 UNC-IDEA 61
22 MNAB 83.86 22 AOC 1.97 22 MSL-SKKU 6.60 22 MNAB2 65
23 Narsil 83.47 23 CVSS 1.98 23 FBI/LMB Freiburg 6.60 23 VIBOT10 M-T1+F 65
24 S2_QM2 83.35 24 BCH_CRL_IMAGINE 2.03 24 CU_DL2 6.60 24 drai 78
25 UB VPML Med 83.31 25 UB VPML Med 2.09 25 ISI-Neonatology 6.62 25 UofL BioImaging 79
26 UofL BioImaging 82.96 26 Narsil 2.27 26 CMIV 6.76 26 SJCE 79
27 CMIV 82.43 27 FreeSurfer* 2.31 27 IDSIA 6.77 27 CVSS 80
28 CVSS 82.35 28 S2_QM2 2.36 28 KM-MRF-MAS 6.79 28 VBM12 80
29 VBM12 82.29 29 drai 2.48 29 CVSS 6.86 29 vicorob UdG T1_F 81
30 vicorob UdG T1_F 82.19 30 LUH-TNT 2.62 30 SPM12_T1* 6.95 30 CMIV 84
31 vicorob UdG T1 82.07 31 CMIV 2.69 31 UNC-IDEA 7.04 31 bigr_neuro 84
32 bigr_neuro 81.46 32 KM-MRF-MAS 2.73 32 MNAB2 7.10 32 S2_QM2 86
33 SPM12_T1_F* 81.17 33 MNAB 2.79 33 SPM12_T1_IR* 7.23 33 vicorob UdG T1 89
34 KM-MRF-MAS 80.85 34 SPM12_T1_F* 2.90 34 S2_QM2 7.23 34 KM-MRF-MAS 94
35 SJCE 80.76 35 SPM12_T1* 2.93 35 VIBOT10 M-T1+F 8.16 35 MNAB 95
36 SPM12_T1* 80.34 36 mlmi2014ss 2.96 36 FSL-PVSeg* 8.40 36 SPM12_T1* 101
37 SPM12_T1_IR_F* 80.15 37 SPM12_T1_IR* 3.01 37 UofL BioImaging 8.61 37 Robarts 102
38 Robarts 79.70 38 SPM12_T1_IR_F* 3.03 38 FSL-Seg* 8.65 38 SPM12_T1_IR* 109
39 SPM12_T1_IR* 79.41 39 vicorob UdG T1_F 3.17 39 Jedi Mind Meld 8.94 39 SPM12_T1_F* 111
40 LUH-TNT 79.35 40 VBM12 3.20 40 MNAB 9.11 40 FreeSurfer* 116
41 FSL-Seg* 78.66 41 vicorob UdG T1 3.31 41 S2_QM 9.28 41 LUH-TNT 117
42 Jedi Mind Meld 77.84 42 bigr_neuro 3.67 42 LNMBrains 9.47 42 SPM12_T1_IR_F* 121
43 FSL-PVSeg* 77.66 43 SJCE 3.71 43 Robarts 9.77 43 FSL-PVSeg* 125
44 FreeSurfer* 77.41 44 ISTB UniBe 3.86 44 SPM12_T1_F* 10.59 44 FSL-Seg* 126
45 S2_QM 76.41 45 Jedi Mind Meld 3.90 45 FreeSurfer* 12.15 45 Jedi Mind Meld 126
46 CSIM-Lab 75.65 46 FSL-PVSeg* 4.29 46 SPM12_T1_IR_F* 13.92 46 S2_QM 134
47 drai 74.53 47 FSL-Seg* 4.35 47 LUH-TNT 15.52 47 mlmi2014ss 135
48 LNMBrains 72.75 48 S2_QM 5.51 48 CSIM-Lab 18.07 48 LNMBrains 139
49 mlmi2014ss 71.70 49 LNMBrains 6.80 49 ISTB UniBe 19.11 49 ISTB UniBe 143
50 ISTB UniBe 68.62 50 CSIM-Lab 7.50 50 mlmi2014ss 24.56 50 CSIM-Lab 144
51 ECOESH 63.36 51 ECOESH 34.07 51 ECOESH 38.34 51 ECOESH 153

White Matter Segmentation Results

Rank DC Rank HD Rank AVD WM Score
1 WTA2 89.74 1 LIVIA_ETS 1.78 1 WTA2 5.20 1 WTA2 4
2 TailHot 89.70 2 WTA2 1.82 2 SPM12_T1_F* 5.21 2 LIVIA_ETS 16
3 LIVIA_ETS 89.46 3 LRDE 1.86 3 MSL-SKKU 5.54 3 LRDE 16
4 CU_DL2 89.46 4 TailHot 1.88 4 AOC 5.72 4 CU_DL 19
5 WTA 89.40 5 CU_DL2 1.93 5 Narsil 5.78 5 CU_DL2 22
6 CU_DL 89.39 6 CU_DL 1.93 6 LRDE 5.83 6 TailHot 22
7 LRDE 89.29 7 FBI/LMB Freiburg 1.95 7 CU_DL 5.84 7 MSL-SKKU 25
8 MSL-SKKU 89.00 8 WTA 2.00 8 PyraMiD-LSTM2 5.93 8 AOC 32
9 MDGRU 88.98 9 MDGRU 2.02 9 vicorob UdG T1_F 5.94 9 PyraMiD-LSTM2 36
10 FBI/LMB Freiburg 88.86 10 UNC-IDEA 2.06 10 SPM12_T1* 5.99 10 FBI/LMB Freiburg 37
11 UNC-IDEA 88.68 11 ISI-Neonatology 2.07 11 BIGR2 6.02 11 WTA 37
12 AOC 88.68 12 PyraMiD-LSTM2 2.07 12 LIVIA_ETS 6.03 12 UNC-IDEA 40
13 ISI-Neonatology 88.66 13 IDSIA 2.08 13 CU_DL2 6.05 13 ISI-Neonatology 46
14 BCH_CRL_IMAGINE 88.63 14 MSL-SKKU 2.11 14 VBM12 6.06 14 BIGR2 50
15 UB VPML Med 88.61 15 MNAB2 2.12 15 vicorob UdG T1 6.21 15 MDGRU 51
16 PyraMiD-LSTM2 88.53 16 AOC 2.19 16 TailHot 6.28 16 VBM12 56
17 STH 88.45 17 BCH_CRL_IMAGINE 2.23 17 SPM12_T1_IR* 6.34 17 IDSIA 57
18 BIGR2 88.42 18 UofL BioImaging 2.24 18 SJCE 6.43 18 vicorob UdG T1_F 64
19 IDSIA 88.33 19 VBM12 2.32 19 UNC-IDEA 6.46 19 KSOM GHMF 66
20 MNAB2 88.04 20 STH 2.34 20 FBI/LMB Freiburg 6.47 20 UB VPML Med 68
21 CVSS 88.00 21 BIGR2 2.36 21 KSOM GHMF 6.59 21 MNAB2 69
22 KSOM GHMF 87.96 22 CMIV 2.43 22 ISI-Neonatology 6.96 22 STH 69
23 VBM12 87.95 23 KSOM GHMF 2.49 23 drai 6.99 23 SPM12_T1_F* 71
24 MNAB 87.95 24 MNAB 2.65 24 WTA 7.02 24 BCH_CRL_IMAGINE 73
25 UofL BioImaging 87.88 25 KM-MRF-MAS 2.70 25 IDSIA 7.05 25 Narsil 74
26 CMIV 87.71 26 UB VPML Med 2.72 26 Robarts 7.06 26 vicorob UdG T1 76
27 vicorob UdG T1_F 87.34 27 CVSS 2.86 27 UB VPML Med 7.13 27 CVSS 77
28 VIBOT10 M-T1+F 87.32 28 vicorob UdG T1_F 2.88 28 FreeSurfer* 7.18 28 CMIV 79
29 bigr_neuro 87.31 29 mlmi2014ss 2.89 29 CVSS 7.20 29 UofL BioImaging 82
30 Narsil 87.10 30 vicorob UdG T1 2.91 30 bigr_neuro 7.31 30 SPM12_T1* 83
31 vicorob UdG T1 87.06 31 LUH-TNT 2.96 31 CMIV 7.34 31 MNAB 83
32 SJCE 86.83 32 SPM12_T1_F* 3.00 32 STH 7.67 32 bigr_neuro 92
33 KM-MRF-MAS 86.73 33 bigr_neuro 3.01 33 MDGRU 7.69 33 SJCE 96
34 LUH-TNT 86.47 34 SPM12_T1* 3.06 34 MNAB2 7.74 34 Robarts 98
35 S2_QM2 86.34 35 FreeSurfer* 3.06 35 MNAB 7.75 35 KM-MRF-MAS 98
36 Robarts 86.15 36 Robarts 3.06 36 S2_QM2 7.91 36 SPM12_T1_IR* 102
37 SPM12_T1_F* 86.03 37 drai 3.23 37 VIBOT10 M-T1+F 8.01 37 FreeSurfer* 103
38 FSL-Seg* 85.98 38 S2_QM2 3.29 38 SPM12_T1_IR_F* 8.40 38 drai 108
39 SPM12_T1* 85.58 39 Narsil 3.31 39 UofL BioImaging 8.74 39 S2_QM2 109
40 FreeSurfer* 85.25 40 SPM12_T1_IR* 3.60 40 KM-MRF-MAS 9.19 40 LUH-TNT 111
41 CSIM-Lab 85.24 41 CSIM-Lab 3.65 41 FSL-Seg* 11.47 41 VIBOT10 M-T1+F 114
42 FSL-PVSeg* 84.84 42 FSL-Seg* 3.65 42 BCH_CRL_IMAGINE 12.38 42 FSL-Seg* 121
43 S2_QM 83.86 43 SPM12_T1_IR_F* 3.84 43 Jedi Mind Meld 13.66 43 mlmi2014ss 121
44 SPM12_T1_IR_F* 83.64 44 FSL-PVSeg* 3.85 44 ISTB UniBe 13.71 44 SPM12_T1_IR_F* 125
45 SPM12_T1_IR* 83.54 45 ISTB UniBe 3.98 45 mlmi2014ss 14.00 45 CSIM-Lab 130
46 ISTB UniBe 82.50 46 SJCE 4.29 46 LUH-TNT 14.20 46 ISTB UniBe 135
47 mlmi2014ss 81.98 47 Jedi Mind Meld 4.52 47 LNMBrains 15.34 47 FSL-PVSeg* 135
48 drai 80.75 48 S2_QM 4.94 48 CSIM-Lab 17.39 48 Jedi Mind Meld 139
49 Jedi Mind Meld 80.64 49 VIBOT10 M-T1+F 6.63 49 FSL-PVSeg* 19.69 49 S2_QM 141
50 LNMBrains 78.32 50 LNMBrains 6.77 50 S2_QM 24.79 50 LNMBrains 147
51 ECOESH 75.82 51 ECOESH 38.46 51 ECOESH 25.27 51 ECOESH 153

Cerebrospinal Fluid Segmentation Results

Rank DC Rank HD Rank AVD CSF Score
1 TailHot 84.86 1 TailHot 2.03 1 WTA2 6.20 1 TailHot 6
2 CU_DL2 84.25 2 IDSIA 2.14 2 AOC 6.51 2 IDSIA 15
3 MDGRU 84.13 3 MDGRU 2.17 3 STH 6.73 3 MDGRU 19
4 CU_DL 83.96 4 CU_DL2 2.19 4 TailHot 6.75 4 CU_DL2 21
5 MSL-SKKU 83.76 5 FBI/LMB Freiburg 2.22 5 MSL-SKKU 6.77 5 MSL-SKKU 22
6 IDSIA 83.72 6 LIVIA_ETS 2.26 6 VBM12 6.80 6 LIVIA_ETS 26
7 FBI/LMB Freiburg 83.47 7 MNAB2 2.27 7 IDSIA 7.09 7 CU_DL 27
8 LIVIA_ETS 83.42 8 LRDE 2.28 8 BCH_CRL_IMAGINE 7.12 8 WTA2 27
9 BCH_CRL_IMAGINE 83.26 9 CU_DL 2.28 9 PyraMiD-LSTM2 7.17 9 STH 27
10 PyraMiD-LSTM2 83.05 10 PyraMiD-LSTM2 2.30 10 vicorob UdG T1 7.17 10 PyraMiD-LSTM2 29
11 UNC-IDEA 82.81 11 STH 2.31 11 vicorob UdG T1_F 7.21 11 FBI/LMB Freiburg 29
12 WTA2 82.77 12 MSL-SKKU 2.32 12 LIVIA_ETS 7.31 12 MNAB2 40
13 STH 82.77 13 UNC-IDEA 2.35 13 MDGRU 7.44 13 AOC 41
14 LRDE 82.44 14 WTA2 2.41 14 CU_DL 7.44 14 LRDE 41
15 MNAB2 82.30 15 ISI-Neonatology 2.65 15 CU_DL2 7.69 15 BCH_CRL_IMAGINE 48
16 KSOM GHMF 82.10 16 Robarts 2.66 16 drai 8.14 16 vicorob UdG T1 49
17 ISI-Neonatology 81.08 17 UofL BioImaging 2.66 17 FBI/LMB Freiburg 8.63 17 UNC-IDEA 49
18 AOC 80.46 18 KSOM GHMF 2.71 18 MNAB2 8.73 18 vicorob UdG T1_F 52
19 Robarts 80.28 19 vicorob UdG T1 2.81 19 LRDE 9.03 19 ISI-Neonatology 54
20 vicorob UdG T1 80.18 20 vicorob UdG T1_F 2.81 20 S2_QM2 9.36 20 UofL BioImaging 61
21 vicorob UdG T1_F 80.17 21 AOC 2.84 21 UofL BioImaging 9.68 21 VBM12 63
22 VIBOT10 M-T1+F 79.73 22 VIBOT10 M-T1+F 2.98 22 ISI-Neonatology 9.77 22 KSOM GHMF 64
23 UofL BioImaging 78.86 23 CMIV 3.00 23 SPM12_T1_F* 10.07 23 S2_QM2 73
24 CMIV 78.57 24 VBM12 3.03 24 SPM12_T1_IR* 10.47 24 Robarts 77
25 BIGR2 78.31 25 S2_QM2 3.05 25 UNC-IDEA 10.52 25 CMIV 80
26 SPM12_T1_IR* 78.25 26 KM-MRF-MAS 3.08 26 FSL-PVSeg* 10.69 26 VIBOT10 M-T1+F 80
27 bigr_neuro 78.16 27 bigr_neuro 3.15 27 KM-MRF-MAS 10.95 27 drai 81
28 S2_QM2 77.97 28 drai 3.17 28 FSL-Seg* 11.83 28 KM-MRF-MAS 83
29 WTA 77.77 29 BIGR2 3.19 29 SPM12_T1_IR_F* 12.21 29 SPM12_T1_IR* 87
30 KM-MRF-MAS 76.97 30 CVSS 3.25 30 KSOM GHMF 12.78 30 bigr_neuro 91
31 SPM12_T1_IR_F* 76.87 31 BCH_CRL_IMAGINE 3.28 31 CSIM-Lab 13.31 31 BIGR2 98
32 UB VPML Med 74.82 32 FSL-Seg* 3.41 32 Narsil 13.54 32 SPM12_T1_IR_F* 99
33 VBM12 74.56 33 FSL-PVSeg* 3.44 33 CMIV 14.31 33 FSL-PVSeg* 100
34 SPM12_T1_F* 74.03 34 Jedi Mind Meld 3.67 34 S2_QM 14.45 34 FSL-Seg* 100
35 Jedi Mind Meld 73.33 35 mlmi2014ss 3.77 35 SJCE 14.99 35 SPM12_T1_F* 100
36 CVSS 72.59 36 S2_QM 3.79 36 VIBOT10 M-T1+F 16.06 36 CVSS 105
37 drai 70.95 37 SPM12_T1_IR* 4.01 37 bigr_neuro 16.47 37 Jedi Mind Meld 107
38 SPM12_T1* 70.69 38 LUH-TNT 4.07 38 Jedi Mind Meld 17.60 38 S2_QM 114
39 mlmi2014ss 70.59 39 SPM12_T1_IR_F* 4.15 39 CVSS 17.94 39 WTA 117
40 FSL-Seg* 69.95 40 FreeSurfer* 4.31 40 LNMBrains 19.87 40 UB VPML Med 121
41 FSL-PVSeg* 69.49 41 UB VPML Med 4.31 41 ISTB UniBe 19.89 41 mlmi2014ss 123
42 LNMBrains 68.80 42 WTA 4.35 42 Robarts 20.36 42 SJCE 124
43 MNAB 68.08 43 SPM12_T1_F* 4.59 43 LUH-TNT 20.40 43 CSIM-Lab 127
44 S2_QM 67.87 44 SJCE 4.78 44 BIGR2 22.76 44 Narsil 128
45 SJCE 67.67 45 MNAB 4.88 45 SPM12_T1* 23.24 45 LUH-TNT 128
46 Narsil 66.57 46 SPM12_T1* 5.34 46 WTA 24.67 46 SPM12_T1* 129
47 LUH-TNT 66.47 47 CSIM-Lab 5.44 47 MNAB 28.56 47 LNMBrains 131
48 FreeSurfer* 65.79 48 ISTB UniBe 6.94 48 UB VPML Med 31.19 48 MNAB 135
49 CSIM-Lab 59.82 49 LNMBrains 7.66 49 mlmi2014ss 33.45 49 FreeSurfer* 139
50 ISTB UniBe 54.69 50 Narsil 13.30 50 ECOESH 36.59 50 ISTB UniBe 139
51 ECOESH 51.48 51 ECOESH 18.75 51 FreeSurfer* 50.27 51 ECOESH 152

Brain (GM and WM segmentations combined) Results

Rank DC Rank HD Rank AVD Brain Score
1 TailHot 95.83 1 LIVIA_ETS 1.99 1 UB VPML Med 2.35 1 LIVIA_ETS 10
2 CU_DL2 95.74 2 LRDE 2.07 2 STH 2.51 2 LRDE 13
3 LIVIA_ETS 95.70 3 TailHot 2.09 3 AOC 2.52 3 TailHot 17
4 CU_DL 95.70 4 WTA2 2.12 4 CMIV 2.58 4 WTA2 19
5 MSL-SKKU 95.67 5 ISI-Neonatology 2.18 5 LRDE 2.62 5 MSL-SKKU 25
6 LRDE 95.65 6 CU_DL 2.19 6 LIVIA_ETS 2.67 6 CU_DL 26
7 ISI-Neonatology 95.65 7 CU_DL2 2.24 7 WTA2 2.68 7 ISI-Neonatology 27
8 WTA2 95.62 8 IDSIA 2.27 8 WTA 2.73 8 CU_DL2 27
9 WTA 95.59 9 MDGRU 2.29 9 PyraMiD-LSTM2 2.73 9 STH 28
10 MDGRU 95.59 10 MSL-SKKU 2.29 10 MSL-SKKU 2.73 10 WTA 29
11 FBI/LMB Freiburg 95.40 11 FBI/LMB Freiburg 2.33 11 VBM12 2.77 11 MDGRU 33
12 BCH_CRL_IMAGINE 95.39 12 WTA 2.35 12 S2_QM2 2.78 12 AOC 37
13 STH 95.37 13 STH 2.38 13 TailHot 2.79 13 PyraMiD-LSTM2 39
14 PyraMiD-LSTM2 95.30 14 MNAB2 2.41 14 MDGRU 2.79 14 IDSIA 46
15 IDSIA 95.29 15 UofL BioImaging 2.41 15 ISI-Neonatology 2.80 15 FBI/LMB Freiburg 49
16 AOC 95.19 16 PyraMiD-LSTM2 2.45 16 CU_DL 2.89 16 UB VPML Med 49
17 MNAB2 95.16 17 UNC-IDEA 2.48 17 Narsil 2.91 17 S2_QM2 53
18 KSOM GHMF 95.13 18 AOC 2.53 18 CU_DL2 2.94 18 MNAB2 57
19 BIGR2 95.13 19 S2_QM2 2.73 19 FSL-Seg* 2.95 19 Narsil 66
20 UNC-IDEA 95.05 20 BIGR2 2.74 20 vicorob UdG T1 2.97 20 CMIV 66
21 UofL BioImaging 94.93 21 KM-MRF-MAS 2.76 21 vicorob UdG T1_F 2.98 21 BCH_CRL_IMAGINE 66
22 S2_QM2 94.90 22 Robarts 2.78 22 BCH_CRL_IMAGINE 3.06 22 VBM12 67
23 VIBOT10 M-T1+F 94.89 23 UB VPML Med 2.80 23 IDSIA 3.10 23 KSOM GHMF 68
24 Narsil 94.78 24 CVSS 2.84 24 KSOM GHMF 3.15 24 BIGR2 68
25 UB VPML Med 94.59 25 Narsil 2.86 25 drai 3.20 25 UofL BioImaging 72
26 VBM12 94.50 26 KSOM GHMF 2.86 26 MNAB2 3.21 26 UNC-IDEA 75
27 CMIV 94.48 27 LUH-TNT 2.90 27 FBI/LMB Freiburg 3.22 27 vicorob UdG T1_F 83
28 MNAB 94.47 28 drai 3.05 28 SPM12_T1* 3.23 28 vicorob UdG T1 84
29 vicorob UdG T1_F 94.27 29 mlmi2014ss 3.23 29 BIGR2 3.24 29 LUH-TNT 95
30 vicorob UdG T1 94.25 30 VBM12 3.36 30 SPM12_T1_IR* 3.38 30 CVSS 96
31 LUH-TNT 94.17 31 FreeSurfer* 3.55 31 Jedi Mind Meld 3.46 31 drai 99
32 CVSS 94.08 32 BCH_CRL_IMAGINE 3.64 32 FSL-PVSeg* 3.46 32 KM-MRF-MAS 99
33 bigr_neuro 93.99 33 vicorob UdG T1_F 3.73 33 bigr_neuro 3.57 33 MNAB 101
34 KM-MRF-MAS 93.97 34 vicorob UdG T1 3.74 34 MNAB 3.76 34 SPM12_T1* 102
35 SPM12_T1_IR* 93.93 35 CMIV 3.82 35 SJCE 3.79 35 FSL-Seg* 104
36 SPM12_T1_F* 93.87 36 ISTB UniBe 4.39 36 UofL BioImaging 3.88 36 VIBOT10 M-T1+F 104
37 SPM12_T1* 93.85 37 SPM12_T1* 4.45 37 LUH-TNT 4.11 37 bigr_neuro 106
38 FSL-PVSeg* 93.59 38 SJCE 4.52 38 UNC-IDEA 4.21 38 SPM12_T1_IR* 106
39 SPM12_T1_IR_F* 93.59 39 MNAB 4.54 39 VIBOT10 M-T1+F 4.84 39 Robarts 113
40 SJCE 93.48 40 bigr_neuro 4.57 40 CVSS 4.91 40 SJCE 113
41 FSL-Seg* 93.30 41 SPM12_T1_IR* 4.63 41 SPM12_T1_IR_F* 5.11 41 FSL-PVSeg* 117
42 S2_QM 93.27 42 VIBOT10 M-T1+F 4.91 42 SPM12_T1_F* 5.33 42 Jedi Mind Meld 117
43 Jedi Mind Meld 93.24 43 Jedi Mind Meld 5.44 43 S2_QM 5.48 43 FreeSurfer* 121
44 Robarts 93.13 44 FSL-Seg* 5.53 44 KM-MRF-MAS 5.58 44 SPM12_T1_F* 123
45 FreeSurfer* 92.32 45 SPM12_T1_F* 5.85 45 FreeSurfer* 6.27 45 mlmi2014ss 125
46 drai 91.97 46 SPM12_T1_IR_F* 5.91 46 LNMBrains 7.37 46 SPM12_T1_IR_F* 126
47 CSIM-Lab 91.20 47 FSL-PVSeg* 6.09 47 Robarts 7.88 47 S2_QM 133
48 mlmi2014ss 90.75 48 S2_QM 6.98 48 mlmi2014ss 12.05 48 ISTB UniBe 136
49 LNMBrains 88.51 49 LNMBrains 8.56 49 CSIM-Lab 15.76 49 LNMBrains 144
50 ISTB UniBe 87.80 50 CSIM-Lab 11.24 50 ISTB UniBe 17.26 50 CSIM-Lab 146
51 ECOESH 78.24 51 ECOESH 35.99 51 ECOESH 32.06 51 ECOESH 153

Intracranial cavity (GM, WM, and CSF segmentations combined) Results

Rank DC Rank HD Rank AVD ICV Score
1 TailHot 98.17 1 MDGRU 2.37 1 UNC-IDEA 0.68 1 MDGRU 9
2 IDSIA 98.15 2 IDSIA 2.44 2 PyraMiD-LSTM2 0.69 2 IDSIA 11
3 MDGRU 98.15 3 FBI/LMB Freiburg 2.51 3 STH 0.69 3 PyraMiD-LSTM2 15
4 CU_DL2 98.10 4 Robarts 2.55 4 BCH_CRL_IMAGINE 0.75 4 UNC-IDEA 16
5 PyraMiD-LSTM2 98.04 5 UNC-IDEA 2.72 5 MDGRU 0.86 5 Robarts 19
6 CU_DL 97.99 6 CU_DL2 2.75 6 Robarts 0.91 6 FBI/LMB Freiburg 19
7 FBI/LMB Freiburg 97.99 7 TailHot 2.78 7 IDSIA 0.95 7 TailHot 19
8 MSL-SKKU 97.96 8 PyraMiD-LSTM2 2.86 8 SPM12_T1_IR* 0.99 8 CU_DL2 25
9 Robarts 97.92 9 LRDE 2.87 9 FBI/LMB Freiburg 1.06 9 STH 27
10 UNC-IDEA 97.92 10 STH 2.92 10 SPM12_T1_IR_F* 1.17 10 MSL-SKKU 41
11 BCH_CRL_IMAGINE 97.82 11 MNAB2 2.95 11 TailHot 1.23 11 LIVIA_ETS 43
12 LIVIA_ETS 97.82 12 LIVIA_ETS 2.96 12 LRDE 1.44 12 LRDE 43
13 SPM12_T1_IR* 97.73 13 WTA2 3.00 13 SPM12_T1_F* 1.46 13 CU_DL 44
14 STH 97.72 14 CU_DL 3.17 14 MSL-SKKU 1.46 14 MNAB2 49
15 SPM12_T1_IR_F* 97.71 15 KSOM GHMF 3.23 15 CU_DL2 1.54 15 vicorob UdG T1 54
16 MNAB2 97.66 16 UofL BioImaging 3.43 16 vicorob UdG T1 1.62 16 SPM12_T1_IR* 56
17 vicorob UdG T1 97.65 17 AOC 3.53 17 VIBOT10 M-T1+F 1.64 17 vicorob UdG T1_F 59
18 vicorob UdG T1_F 97.64 18 S2_QM2 3.72 18 vicorob UdG T1_F 1.65 18 AOC 60
19 VIBOT10 M-T1+F 97.63 19 MSL-SKKU 3.75 19 LIVIA_ETS 1.67 19 VIBOT10 M-T1+F 60
20 WTA2 97.61 20 CMIV 3.76 20 AOC 1.73 20 WTA2 60
21 KSOM GHMF 97.56 21 vicorob UdG T1 3.85 21 ISI-Neonatology 1.74 21 BCH_CRL_IMAGINE 62
22 LRDE 97.53 22 drai 3.89 22 MNAB2 1.77 22 KSOM GHMF 62
23 AOC 97.22 23 vicorob UdG T1_F 3.90 23 UofL BioImaging 1.81 23 UofL BioImaging 65
24 ISI-Neonatology 96.92 24 VIBOT10 M-T1+F 3.91 24 CU_DL 1.83 24 SPM12_T1_IR_F* 69
25 CMIV 96.79 25 BIGR2 3.93 25 drai 1.85 25 ISI-Neonatology 72
26 UofL BioImaging 96.71 26 bigr_neuro 3.93 26 KSOM GHMF 1.95 26 S2_QM2 75
27 SPM12_T1_F* 96.59 27 ISI-Neonatology 4.29 27 WTA2 1.98 27 drai 79
28 S2_QM2 96.44 28 CVSS 4.87 28 VBM12 2.37 28 SPM12_T1_F* 83
29 bigr_neuro 96.33 29 VBM12 4.89 29 S2_QM2 2.38 29 CMIV 83
30 KM-MRF-MAS 96.10 30 KM-MRF-MAS 5.08 30 CVSS 2.55 30 bigr_neuro 89
31 BIGR2 96.00 31 FSL-Seg* 5.29 31 KM-MRF-MAS 2.99 31 VBM12 90
32 drai 95.99 32 FSL-PVSeg* 5.29 32 FSL-Seg* 3.37 32 KM-MRF-MAS 91
33 VBM12 95.95 33 WTA 5.32 33 FSL-PVSeg* 3.37 33 BIGR2 95
34 mlmi2014ss 95.84 34 S2_QM 6.49 34 bigr_neuro 3.46 34 CVSS 96
35 WTA 95.34 35 SPM12_T1_IR* 6.51 35 mlmi2014ss 3.60 35 FSL-Seg* 104
36 SPM12_T1* 95.28 36 FreeSurfer* 6.62 36 Narsil 3.69 36 FSL-PVSeg* 105
37 UB VPML Med 94.83 37 UB VPML Med 6.63 37 S2_QM 4.01 37 WTA 111
38 CVSS 94.62 38 LUH-TNT 6.85 38 CMIV 4.87 38 S2_QM 113
39 Jedi Mind Meld 94.26 39 MNAB 7.06 39 BIGR2 5.18 39 mlmi2014ss 114
40 FSL-PVSeg* 94.24 40 SJCE 7.31 40 SPM12_T1* 5.53 40 SPM12_T1* 118
41 FSL-Seg* 94.24 41 CSIM-Lab 8.09 41 SJCE 6.77 41 UB VPML Med 118
42 S2_QM 93.71 42 SPM12_T1* 8.09 42 Jedi Mind Meld 6.82 42 SJCE 124
43 SJCE 93.66 43 SPM12_T1_F* 8.21 43 WTA 6.86 43 LUH-TNT 128
44 LUH-TNT 93.18 44 SPM12_T1_IR_F* 8.23 44 UB VPML Med 7.67 44 Jedi Mind Meld 129
45 MNAB 92.52 45 mlmi2014ss 8.69 45 LNMBrains 7.88 45 MNAB 131
46 Narsil 92.49 46 ISTB UniBe 9.84 46 LUH-TNT 9.14 46 FreeSurfer* 132
47 FreeSurfer* 92.39 47 BCH_CRL_IMAGINE 11.34 47 MNAB 9.71 47 Narsil 133
48 CSIM-Lab 92.34 48 Jedi Mind Meld 20.34 48 CSIM-Lab 10.06 48 CSIM-Lab 137
49 LNMBrains 89.17 49 LNMBrains 20.36 49 FreeSurfer* 10.19 49 LNMBrains 143
50 ISTB UniBe 88.53 50 ECOESH 21.43 50 ISTB UniBe 17.52 50 ISTB UniBe 146
51 ECOESH 78.14 51 Narsil 23.65 51 ECOESH 30.13 51 ECOESH 152

DC = Dice Coefficient, HD = 95th percentile of Hausdorff Distance, AVD = Absolute Volume Difference
Score = Rank DC + Rank HD + Rank AVD





© 2013 Image Sciences Institute.