In these sets of experiments, we repeated the same procedure described in the previous post with only one difference. So, please see the details in the previous post. The difference is the way that we partitioned the patches into training and validation sets. We obtained patches from 8 simulation images from Field2. Then, randomly pick 1/8 of patches and put them into validation set. The remaining patches formed the training set.
Note: Best and Worst cases are chosen by comparing the average bubble error in a patch.
Quantitative Metrics: In order to compare success of different experiments, we decide the use the followings:
Quantitative Metric1 = \[ L1Loss(z**f_0 – x**f_0) \]
Quantitative Metric2 = \[ MSELossLoss(z**f_0 – x**f_0) \]
where f_0 is a gaussian kernel with sigma =1.
Note: Both f and f_0 are in pixel coordinates.
Experiment 1: learning rate=1e-5, sigma of kernel f=1, regularizer parameter=0.01



Average quantitative metric1 is 19.645459
Average metric1 per bubble is 0.7747
Average quantitative metric2 is 0.981560
Average metric2 per bubble is 0.0384

Worst Case in terms of Per Bubble Error: Center Location(x,z) = 34.53 mm and 10.85 mm, Average Metric1 per bubble: 1.31, Average Metric2 per bubble: : 0.0756




Best Case in terms of Per Bubble Error:Center Location(x,z) = 19.75 mm and 44.37 mm , Quantitative Metric1: 0.511, Quantitative Metric2: 0.0173




Experiment 2: learning rate=1e-5, sigma of kernel f=1, regularizer parameter=0.005



Average quantitative metric1 is 20.352445
Average metric1 per bubble is 0.802987
Average quantitative metric2 is 0.985063
Average metric2 per bubble is 0.038526

Worst Case in terms of Per Bubble Error:Center Location(x,z) = 29.61 mm and 9.87 mm , Average Metric1 per bubble: : 1.235, Average Metric2 per bubble: : 0.0755




Best Case in terms of Per Bubble Error:Center Location(x,z) = 24.68 mm and 33.53 mm , Average Metric1 per bubble: : 0.538, Average Metric2 per bubble: : 0.0134




Experiment 3: learning rate=1e-5, sigma of kernel f=1.5, regularizer parameter=0.01



Average quantitative metric1 is 18.815349
Average metric1 per bubble is 0.742511
Average quantitative metric2 is 0.936104
Average metric2 per bubble is 0.036656

Worst Case in terms of Per Bubble Error:Center Location(x,z) = 2.50 mm and 13.81 mm , Average Metric1 per bubble: : 1.1237, Average Metric2 per bubble: : 0.071355




Best Case in terms of Per Bubble Error:Center Location(x,z) = 12.36 mm and 41.41 mm , Average Metric1 per bubble: : 0.522, Average Metric2 per bubble: : 0.0179




Experiment 4: learning rate=1e-5, sigma of kernel f=1.5, regularizer parameter=0.005



Average quantitative metric1 is 19.907855
Average metric1 per bubble is 0.785045
Average quantitative metric2 is 0.928157
Average metric2 per bubble is 0.036292

Worst Case in terms of Per Bubble Error:Center Location(x,z) = 2.50 mm and 17.76 mm , Average Metric1 per bubble: : 1.175, Average Metric2 per bubble: : 0.068




Best Case in terms of Per Bubble Error:Center Location(x,z) = 24.68 mm and 33.53 mm , Average Metric1 per bubble: : 0.519, Average Metric2 per bubble: : 0.0144




Experiment 5: learning rate=1e-5, sigma of kernel f=2, regularizer parameter=0.01



Average quantitative metric1 is 19.071611
Average metric1 per bubble is 0.753008
Average quantitative metric2 is 1.000624
Average metric2 per bubble is 0.039267

Worst Case in terms of Per Bubble Error:Center Location(x,z) = 2.50 mm and 13.81 mm , Average Metric1 per bubble: 1.045525, Average Metric2 per bubble: 0.065981




Best Case in terms of Per Bubble Error:Center Location(x,z) = 17.27 mm and 27.61 mm , Average Metric1 per bubble: 0.571219, Average Metric2 per bubble: 0.0237




Experiment 6: learning rate=1e-5, sigma of kernel f=2, regularizer parameter=0.005



Average quantitative metric1 is 19.766309
Average metric1 per bubble is 0.780344
Average quantitative metric2 is 0.954419
Average metric2 per bubble is 0.037464

Worst Case in terms of Per Bubble Error:Center Location(x,z) = 34.53 mm and 22.68 mm , Average Metric1 per bubble: 1.182416, Average Metric2 per bubble: 0.063596




Best Case in terms of Per Bubble Error:Center Location(x,z) = 22.21 mm and 37.47 mm , Average Metric1 per bubble: 0.561, Average Metric1 per bubble: 0.01984




The above results are for disabled random shuffling in training phase. The below table results are obtained using random shuffle.
sigma of f | \(\lambda \) | learning rate | number of epochs | batch size | Q1 | Q2 | Q1 per bubble | Q2 per bubble | % gap between training and validation | |
exp1 | 1 | 0.01 | 1e-5 | 300 | 1 | 16.241709 | 0.817596 | 0.638174 | 0.031777 | 112 |
exp2 | 1 | 0.005 | 1e-5 | 300 | 1 | 16.662992 | 0.774843 | 0.657008 | 0.030192 | 151 |
exp3 | 1.5 | 0.01 | 1e-5 | 300 | 1 | 16.189029 | 0.805056 | 0.637 | 0.031413 | 48.5 |
exp4 | 1.5 | 0.005 | 1e-5 | 300 | 1 | 16.565583 | 0.772228 | 0.652627 | 0.030147 | 91.3 |
exp5 | 2 | 0.01 | 1e-5 | 300 | 1 | 17.008482 | 0.878161 | 0.669787 | 0.034329 | 22 |
exp6 | 2 | 0.005 | 1e-5 | 300 | 1 | 18.784188 | 0.928413 | 0.742285 | 0.036475 | 54.6 |