In these sets of experiments, we repeated the same procedure described in the previous post with only one difference. So, please see the details in the previous post. The difference is the way that we partitioned the patches into training and validation sets. We obtained patches from 8 simulation images from Field2. Then, randomly pick 1/8 of patches and put them into validation set. The remaining patches formed the training set.
Note: Best and Worst cases are chosen by comparing the average bubble error in a patch.
Quantitative Metrics: In order to compare success of different experiments, we decide the use the followings:
Quantitative Metric1 = \[ L1Loss(z**f_0 – x**f_0) \]
Quantitative Metric2 = \[ MSELossLoss(z**f_0 – x**f_0) \]
where f_0 is a gaussian kernel with sigma =1.
Note: Both f and f_0 are in pixel coordinates.
Experiment 1: learning rate=1e-5, sigma of kernel f=1, regularizer parameter=0.01
Average quantitative metric1 is 19.645459
Average metric1 per bubble is 0.7747
Average quantitative metric2 is 0.981560
Average metric2 per bubble is 0.0384
Worst Case in terms of Per Bubble Error: Center Location(x,z) = 34.53 mm and 10.85 mm, Average Metric1 per bubble: 1.31, Average Metric2 per bubble: : 0.0756
Best Case in terms of Per Bubble Error:Center Location(x,z) = 19.75 mm and 44.37 mm , Quantitative Metric1: 0.511, Quantitative Metric2: 0.0173
Experiment 2: learning rate=1e-5, sigma of kernel f=1, regularizer parameter=0.005
Average quantitative metric1 is 20.352445
Average metric1 per bubble is 0.802987
Average quantitative metric2 is 0.985063
Average metric2 per bubble is 0.038526
Worst Case in terms of Per Bubble Error:Center Location(x,z) = 29.61 mm and 9.87 mm , Average Metric1 per bubble: : 1.235, Average Metric2 per bubble: : 0.0755
Best Case in terms of Per Bubble Error:Center Location(x,z) = 24.68 mm and 33.53 mm , Average Metric1 per bubble: : 0.538, Average Metric2 per bubble: : 0.0134
Experiment 3: learning rate=1e-5, sigma of kernel f=1.5, regularizer parameter=0.01
Average quantitative metric1 is 18.815349
Average metric1 per bubble is 0.742511
Average quantitative metric2 is 0.936104
Average metric2 per bubble is 0.036656
Worst Case in terms of Per Bubble Error:Center Location(x,z) = 2.50 mm and 13.81 mm , Average Metric1 per bubble: : 1.1237, Average Metric2 per bubble: : 0.071355
Best Case in terms of Per Bubble Error:Center Location(x,z) = 12.36 mm and 41.41 mm , Average Metric1 per bubble: : 0.522, Average Metric2 per bubble: : 0.0179
Experiment 4: learning rate=1e-5, sigma of kernel f=1.5, regularizer parameter=0.005
Average quantitative metric1 is 19.907855
Average metric1 per bubble is 0.785045
Average quantitative metric2 is 0.928157
Average metric2 per bubble is 0.036292
Worst Case in terms of Per Bubble Error:Center Location(x,z) = 2.50 mm and 17.76 mm , Average Metric1 per bubble: : 1.175, Average Metric2 per bubble: : 0.068
Best Case in terms of Per Bubble Error:Center Location(x,z) = 24.68 mm and 33.53 mm , Average Metric1 per bubble: : 0.519, Average Metric2 per bubble: : 0.0144
Experiment 5: learning rate=1e-5, sigma of kernel f=2, regularizer parameter=0.01
Average quantitative metric1 is 19.071611
Average metric1 per bubble is 0.753008
Average quantitative metric2 is 1.000624
Average metric2 per bubble is 0.039267
Worst Case in terms of Per Bubble Error:Center Location(x,z) = 2.50 mm and 13.81 mm , Average Metric1 per bubble: 1.045525, Average Metric2 per bubble: 0.065981
Best Case in terms of Per Bubble Error:Center Location(x,z) = 17.27 mm and 27.61 mm , Average Metric1 per bubble: 0.571219, Average Metric2 per bubble: 0.0237
Experiment 6: learning rate=1e-5, sigma of kernel f=2, regularizer parameter=0.005
Average quantitative metric1 is 19.766309
Average metric1 per bubble is 0.780344
Average quantitative metric2 is 0.954419
Average metric2 per bubble is 0.037464
Worst Case in terms of Per Bubble Error:Center Location(x,z) = 34.53 mm and 22.68 mm , Average Metric1 per bubble: 1.182416, Average Metric2 per bubble: 0.063596
Best Case in terms of Per Bubble Error:Center Location(x,z) = 22.21 mm and 37.47 mm , Average Metric1 per bubble: 0.561, Average Metric1 per bubble: 0.01984
The above results are for disabled random shuffling in training phase. The below table results are obtained using random shuffle.
| sigma of f | \(\lambda \) | learning rate | number of epochs | batch size | Q1 | Q2 | Q1 per bubble | Q2 per bubble | % gap between training and validation | |
| exp1 | 1 | 0.01 | 1e-5 | 300 | 1 | 16.241709 | 0.817596 | 0.638174 | 0.031777 | 112 |
| exp2 | 1 | 0.005 | 1e-5 | 300 | 1 | 16.662992 | 0.774843 | 0.657008 | 0.030192 | 151 |
| exp3 | 1.5 | 0.01 | 1e-5 | 300 | 1 | 16.189029 | 0.805056 | 0.637 | 0.031413 | 48.5 |
| exp4 | 1.5 | 0.005 | 1e-5 | 300 | 1 | 16.565583 | 0.772228 | 0.652627 | 0.030147 | 91.3 |
| exp5 | 2 | 0.01 | 1e-5 | 300 | 1 | 17.008482 | 0.878161 | 0.669787 | 0.034329 | 22 |
| exp6 | 2 | 0.005 | 1e-5 | 300 | 1 | 18.784188 | 0.928413 | 0.742285 | 0.036475 | 54.6 |
