On uncertainty estimation in active learning for image segmentation

This repository provides the implementation for our paper On uncertainty estimation in active learning for image segmentation (Bo Li, Tommy Sonne Alstrøm) [NEED TO ADD A LINK]. We experimentally show that the region based active learning strategy can lead to higher segmentation accuracy and better calibrated model much faster than full image acquisition:

Installation and preparation

1. Clone and enter this repo:

   git clone https://lab.compute.dtu.dk/s161488/region_active_learning.git
   cd Region_Active_Learning

2. Prepare the Dataset and pertained resnet-50 ckpt

   ./requirement.sh

3. Create a virtual env with the required packages

   conda env create --act active_learning.yaml
   source activate act

Evaluate the model

In order to evaluate the model at each acquisition step, run

python3 -c 'import Test as te;te.running_test_for_single_acquisition_step(model_dir)'
Args:
  model_dir: the directory that saves the model ckpt

Train the model

• For full image based active learning, run

  python3 -c 'import Train_Active_Full_Im as tafi;tafi.running_loop_active_learning_full_image(stage)'
  Args:
    stage: int, 0:random, 1:VarRatio, 2:Entropy, 3:BALD

• For region based active learning, run

  python3 -c 'import Train_Active_Region_Im as tari;tari.running_loop_active_learning_region(stage)'
  Args:
    stage: int, 0:random, 1:VarRatio, 2:Entropy, 3:BALD

Reproduce figures

In order to reproduce the figures in the paper, run

<<<<<<< HEAD
./produce_figure.sh
=======
# Download the statistics
cd Exp_Stat
wget https:
tar -xzvf calibration_score.tar.gz
cd ..
cd eval_calibration
python3 visualize_calibration_score.py --save True
>>>>>>> 86802244ca958cf2c36e6ac8940fac4b6c09d8f4