This work is continuation of previous blog named as ULM Through DL 8. The only difference is that in training phase, we used random shuffling.
Summary Table 1:
| sigma of f | \(\lambda \) | learning rate | number of epochs | batch size | Q1 | Q2 | Q1 per bubble | Q2 per bubble | % gap between training and validation | |
| exp1 | 1 | 0.01 | 1e-5 | 200 | 1 | 7.881245 | 0.280391 | 0.309668 | 0.010842 | 27.3 |
| exp2 | 1 | 0.005 | 1e-5 | 200 | 1 | 7.557629 | 0.242156 | 0.297928 | 0.009413 | 37.2 |
| exp3 | 1.5 | 0.01 | 1e-5 | 200 | 1 | 10.248635 | 0.379669 | 0.404081 | 0.014834 | 11.6 |
| exp4 | 1.5 | 0.005 | 1e-5 | 200 | 1 | 9.826689 | 0.310750 | 0.387958 | 0.012168 | 19.3 |
| exp5 | 2 | 0.01 | 1e-5 | 200 | 1 | 12.216727 | 0.517551 | 0.482225 | 0.020331 | 4.6 |
| exp6 | 2 | 0.005 | 1e-5 | 200 | 1 | 11.252816 | 0.393360 | 0.444525 | 0.015460 | 9.2 |
Summary Table 2:
After completing 200 epochs, we saved the weights and reload to continue training. The results are given in below table.
| sigma of f | \(\lambda \) | learning rate | number of epochs | batch size | Q1 | Q2 | Q1 per bubble | Q2 per bubble | % gap between training and validation | |
| exp1 | 1 | 0.01 | 1e-5 | 500 | 1 | 6.275147 | 0.232807 | 0.246134 | 0.008981 | 39.1 |
| exp2 | 1 | 0.005 | 1e-5 | 500 | 1 | 5.723430 | 0.185324 | 0.224921 | 0.007168 | 50.5 |
| exp3 | 1.5 | 0.01 | 1e-5 | 400 | 1 | 8.815558 | 0.311788 | 0.347501 | 0.012197 | 11.6 |
| exp4 | 1.5 | 0.005 | 1e-5 | 400 | 1 | 8.312071 | 0.246856 | 0.328369 | 0.009661 | 23.7 |
| exp5 | 2 | 0.01 | 1e-5 | 400 | 1 | 11.149501 | 0.440528 | 0.450922 | 0.017738 | 5.5 |
| exp6 | 2 | 0.005 | 1e-5 | 400 | 1 | 9.975222 | 0.393808 | 0.334005 | 0.013113 | 11 |
Note: Experiments results that are showed below are related to Summary Table 1.
Experiment 1: learning rate=1e-5, sigma of kernel f=1, regularizer parameter=0.01
Percentage gap between training and validation error ( \( \frac{validation errror – training error}{training error}*100 \) ) = 27.3
Average quantitative metric1 is 7.881245
Average metric1 per bubble is 0.309668
Average quantitative metric2 is 0.280391
Average metric2 per bubble is 0.010842
Worst Case:Center Location(x,z) = 32.070499999999996 mm and 48.8026 mm , Average Metric1 per bubble: 0.915509, Average Metric2 per bubble: : 0.063432
Best Case:Center Location(x,z) =19.750500000000002 mm, 47.3242 mm, Average Metric1 per bubble: 0.164941, Average Metric2 per bubble: : 0.001859
Experiment 2: learning rate=1e-5, sigma of kernel f=1, regularizer parameter=0.005
Percentage gap between training and validation error ( \( \frac{validation errror – training error}{training error}*100 \) ) = 37.2
Average quantitative metric1 is 7.557629
Average metric1 per bubble is 0.297928
Average quantitative metric2 is 0.242156
Average metric2 per bubble is 0.009413
Worst Case:Center Location(x,z) =29.6065 mm and 48.8026 mm , Average Metric1 per bubble: 0.893718, Average Metric2 per bubble: : 0.057056
Best Case:Center Location(x,z) = 13.590499999999999 mm and 47.817 mm , Average Metric1 per bubble: 0.187383, Average Metric2 per bubble: : 0.002258
Experiment 3: learning rate=1e-5, sigma of kernel f=1.5, regularizer parameter=0.01
Percentage gap between training and validation error ( \( \frac{validation errror – training error}{training error}*100 \) ) = 11.6
Average quantitative metric1 is 10.248635
Average metric1 per bubble is 0.404081
Average quantitative metric2 is 0.379669
Average metric2 per bubble is 0.014834
Worst Case:Center Location(x,z) = 29.6065 mm and 48.8026 mm , Average Metric1 per bubble: 0.914298, Average Metric2 per bubble: : 0.060615
Best Case:Center Location(x,z) = 23.4465 mm and 47.3242 mm, Average Metric1 per bubble: 0.291016, Average Metric2 per bubble: : 0.006595
Experiment 4: learning rate=1e-5, sigma of kernel f=1.5, regularizer parameter=0.005
Percentage gap between training and validation error ( \( \frac{validation errror – training error}{training error}*100 \) ) = 19.3
Average quantitative metric1 is 9.826689
Average metric1 per bubble is 0.387958
Average quantitative metric2 is 0.310750
Average metric2 per bubble is 0.012168
Worst Case:Center Location(x,z) = 32.070499999999996 mm and 48.8026 mm , Average Metric1 per bubble: 0.884925, Average Metric2 per bubble: : 0.050834
Best Case:Center Location(x,z) = 13.590499999999999 mm and 38.4538 mm , Average Metric1 per bubble: 0.274615, Average Metric2 per bubble: : 0.004607
Experiment 5: learning rate=1e-5, sigma of kernel f=2, regularizer parameter=0.01
Percentage gap between training and validation error ( \( \frac{validation errror – training error}{training error}*100 \) ) = 4.6
Average quantitative metric1 is 12.216727
Average metric1 per bubble is 0.482225
Average quantitative metric2 is 0.517551
Average metric2 per bubble is 0.020331
Worst Case:Center Location(x,z) = 29.6065 mm and 48.802 mm , Average Metric1 per bubble: 0.876782, Average Metric2 per bubble: 0.058363
Best Case:Center Location(x,z) = 13.590499999999999 mm and 38.4538 mm , Average Metric1 per bubble: 0.378525, Average Metric2 per bubble: : 0.012077
Experiment 6: learning rate=1e-5, sigma of kernel f=2, regularizer parameter=0.005
Percentage gap between training and validation error ( \( \frac{validation errror – training error}{training error}*100 \) ) = 9.2
Average quantitative metric1 is 11.252816
Average metric1 per bubble is 0.444525
Average quantitative metric2 is 0.393360
Average metric2 per bubble is 0.015460
Worst Case:Center Location(x,z) = 29.6065 mm and 48.8026 mm , Average Metric1 per bubble: 0.941545, Average Metric2 per bubble: : 0.057581
Best Case:Center Location(x,z) = 14.8225 mm and 37.961 mm , Average Metric1 per bubble: 0.335283, Average Metric2 per bubble: : 0.008589
