|Home Details Workshop Register Download Submit Results Organizers|
The MRBrainS challenge
Many algorithms for segmenting brain structures in MRI scans have been proposed over the years. Especially in such a well-established research area, there is a tremendous need for fair comparison of these methods with respect to accuracy and robustness. Publicly available databases like the ‘Alzheimer’s Disease Neuroimaging Initiative’ (ADNI) and the ‘Internet Brain Segmentation Repository’ (IBSR) are great initiatives and promote the use of publicly available data for automatic segmentation algorithm comparison, but either lack full manual segmentations (ADNI) or provide low-field (1.5T) single sequence (T1-weighted) MRI data (IBSR). Although there is an increasing awareness of the importance of comparing different algorithms on the same data, many methods are still compared to previous versions of the same type of algorithm on privately held data. This complicates the choice for a certain brain segmentation method among a wide variety of available methods. Furthermore, high-field (3T) multi-sequence MRI data is increasingly available in the clinic, while some available methods were developed for 1.5T single sequence T1-weighted MRI data. Comparing these methods to multi-sequence approaches on high-field (3T) data is of great interest to the medical image processing community.
The aim of this challenge is to compare (semi-)automatic algorithms for segmentation of gray matter, white matter and cerebrospinal fluid on 3T MRI scans of the brain. Since participants to the challenge apply their own methods to the data, there is no bias toward one particular method, and parameters are optimally tuned to achieve the best possible performance. For the challenge, twenty fully annotated multi-sequence (T1-weighted, T1-weighted inversion recovery and FLAIR) 3T MRI brain scans are available. More information on the data can be found here. We encourage both authors of new methods as well as authors of already published brain segmentation methods to participate in the challenge and tune their algorithms in the best possible way on the training data, to enable a fair comparison of the methods. We welcome both multi- and single-sequence (only T1-weighted scan) approaches.
If you wish to participate in the MRBrainS challenge, read the details on this page and follow the steps below:
We kept the segmentation task as simple as possible, to allow more algorithms to participate. Participants should segment gray matter, white matter and cerebrospinal fluid. Algorithms that are able to segment substructures, such as cortical gray matter and basal ganglia, should merge these results and label them as gray matter. This is described in further detail in the Test data section below. In the manual segmentations provided for training, the substructures are labeled separately to allow algorithms that segment substructures to use these labels. Authors with algorithms that only use labels for gray matter, white matter and cerebrospinal fluid, should merge these labels themselves. This is described in further detail in the Training data section below. Both multi-sequence as well as single-sequence (using only the T1-weighted scan) approaches are welcome to participate. We will distinguish between these approaches in the results section.
Twenty fully annotated multi-sequence (T1-weighted, T1-weighted inversion recovery and FLAIR) 3T MRI brain scans are available. These scans have been acquired at the UMC Utrecht (the Netherlands) of patients with diabetes and matched controls (with increased cardiovascular risk) with varying degrees of atrophy and white matter lesions (age > 50).
For each patient, the following sequences are provided:
Five datasets are provided with manual segmentations to use as training data to tune your segmentation algorithm. Manual segmentations (ground truth) were drawn on the thick-slice scans (3mm slice thickness), using an in-house developed manual segmentation tool based on the contour segmentation objects (CSO) tool available in Mevislab. A freehand spline drawing technique was used to segment all structures in the brain. The outline of each structure was delineated, starting with the innermost structures. By iteratively subtracting delineations to create holes, binary images were created for each structure. Segmentations were performed in a darkened room with optimal viewing conditions. All segmentations were inspected for correctness by an expert not involved in the segmentation procedure and corrections were made if needed. A third expert approved all final segmentations.
The following structures are manually segmented and will be available for training:
Notes on the manual segmentations
Your submission should be one image with the voxels labeled as described above. If your algorithm segments substructures (e.g. basal ganglia or ventricles), you should assign them to one of the above classes (1-3). The brainstem and cerebellum will be excluded from the evaluation. Your submission should be written in the ITK MetaImage format (mhd and raw files) or Nifti format (*.nii for matlab users). The ITK MetaImage format consists of a text file with "mhd" extension that contains the ASCII header information and a binary file with "raw" extension that contains the voxel data.
Please note that your result file should have the thick slice resolution (voxel size: 0.958mm x 0.958mm x 3.0mm), since the ground truth is only available for the thick slice data!results page.
The collection of the data, the organization of the MRBrainS challenge and the maintenance of this website require a large effort. In the spirit of cooperative scientific progress, we are committed to maintaining this site as a public repository of benchmark results for segmentation of gray matter, white matter and cerebrospinal fluid in MR brain images. However, we ask everyone who uses this site to respect the "Terms of Participation" described below.
These terms amount to a simple tit for tat: we actively encourage anyone to use the data for testing brain segmentation algorithms. In return, we ask you to submit the results of your method and send us a document that describes your method. The score of your algorithm and your description will be made publicly available in the results section of this website.
We do not claim any ownership or rights to the algorithms or uploaded documents, and do not want to create any obstacles for publishing methods that use the MRBrainS data. However, by participating in the MRBrainS13 challenge (registering a team and downloading the data), you acknowledge that you have read, understood, and agree to be bound by the following "Terms of Participation":