This work is continuation of previous blog named as ULM Through DL 8. The only difference is that in training phase, we used random shuffling.

**Summary Table 1:**

sigma of f | \(\lambda \) | learning rate | number of epochs | batch size | Q1 | Q2 | Q1 per bubble | Q2 per bubble | % gap between training and validation | |

exp1 | 1 | 0.01 | 1e-5 | 200 | 1 | 7.881245 | 0.280391 | 0.309668 | 0.010842 | 27.3 |

exp2 | 1 | 0.005 | 1e-5 | 200 | 1 | 7.557629 | 0.242156 | 0.297928 | 0.009413 | 37.2 |

exp3 | 1.5 | 0.01 | 1e-5 | 200 | 1 | 10.248635 | 0.379669 | 0.404081 | 0.014834 | 11.6 |

exp4 | 1.5 | 0.005 | 1e-5 | 200 | 1 | 9.826689 | 0.310750 | 0.387958 | 0.012168 | 19.3 |

exp5 | 2 | 0.01 | 1e-5 | 200 | 1 | 12.216727 | 0.517551 | 0.482225 | 0.020331 | 4.6 |

exp6 | 2 | 0.005 | 1e-5 | 200 | 1 | 11.252816 | 0.393360 | 0.444525 | 0.015460 | 9.2 |

**Summary Table 2:**

After completing 200 epochs, we saved the weights and reload to continue training. The results are given in below table.

sigma of f | \(\lambda \) | learning rate | number of epochs | batch size | Q1 | Q2 | Q1 per bubble | Q2 per bubble | % gap between training and validation | |

exp1 | 1 | 0.01 | 1e-5 | 500 | 1 | 6.275147 | 0.232807 | 0.246134 | 0.008981 | 39.1 |

exp2 | 1 | 0.005 | 1e-5 | 500 | 1 | 5.723430 | 0.185324 | 0.224921 | 0.007168 | 50.5 |

exp3 | 1.5 | 0.01 | 1e-5 | 400 | 1 | 8.815558 | 0.311788 | 0.347501 | 0.012197 | 11.6 |

exp4 | 1.5 | 0.005 | 1e-5 | 400 | 1 | 8.312071 | 0.246856 | 0.328369 | 0.009661 | 23.7 |

exp5 | 2 | 0.01 | 1e-5 | 400 | 1 | 11.149501 | 0.440528 | 0.450922 | 0.017738 | 5.5 |

exp6 | 2 | 0.005 | 1e-5 | 400 | 1 | 9.975222 | 0.393808 | 0.334005 | 0.013113 | 11 |

** Note:** Experiments results that are showed below are related to Summary Table 1.

** Experiment 1: **learning rate=1e-5, sigma of kernel f=1, regularizer parameter=0.01

**Percentage gap between training and validation error** ( \( \frac{validation errror – training error}{training error}*100 \) ) = 27.3

**Average quantitative metric1** is 7.881245

**Average metric1 per bubble** is 0.309668

**Average quantitative metric2** is 0.280391

**Average metric2 per bubble** is 0.010842

Worst Case:Center Location(x,z) = 32.070499999999996 mm and 48.8026 mm , Average Metric1 per bubble: 0.915509, Average Metric2 per bubble: : 0.063432

Best Case:Center Location(x,z) =19.750500000000002 mm, 47.3242 mm, Average Metric1 per bubble: 0.164941, Average Metric2 per bubble: : 0.001859

** Experiment 2: **learning rate=1e-5, sigma of kernel f=1, regularizer parameter=0.005

**Percentage gap between training and validation error** ( \( \frac{validation errror – training error}{training error}*100 \) ) = 37.2

**Average quantitative metric1** is 7.557629

**Average metric1 per bubble** is 0.297928

**Average quantitative metric2** is 0.242156

**Average metric2 per bubble** is 0.009413

Worst Case:Center Location(x,z) =29.6065 mm and 48.8026 mm , Average Metric1 per bubble: 0.893718, Average Metric2 per bubble: : 0.057056

Best Case:Center Location(x,z) = 13.590499999999999 mm and 47.817 mm , Average Metric1 per bubble: 0.187383, Average Metric2 per bubble: : 0.002258

** Experiment 3: **learning rate=1e-5, sigma of kernel f=1.5, regularizer parameter=0.01

**Percentage gap between training and validation error** ( \( \frac{validation errror – training error}{training error}*100 \) ) = 11.6

**Average quantitative metric1** is 10.248635

**Average metric1 per bubble** is 0.404081

**Average quantitative metric2** is 0.379669

**Average metric2 per bubble** is 0.014834

Worst Case:Center Location(x,z) = 29.6065 mm and 48.8026 mm , Average Metric1 per bubble: 0.914298, Average Metric2 per bubble: : 0.060615

Best Case:Center Location(x,z) = 23.4465 mm and 47.3242 mm, Average Metric1 per bubble: 0.291016, Average Metric2 per bubble: : 0.006595

** Experiment 4: **learning rate=1e-5, sigma of kernel f=1.5, regularizer parameter=0.005

**Percentage gap between training and validation error** ( \( \frac{validation errror – training error}{training error}*100 \) ) = 19.3

**Average quantitative metric1** is 9.826689

**Average metric1 per bubble** is 0.387958

**Average quantitative metric2** is 0.310750

**Average metric2 per bubble** is 0.012168

Worst Case:Center Location(x,z) = 32.070499999999996 mm and 48.8026 mm , Average Metric1 per bubble: 0.884925, Average Metric2 per bubble: : 0.050834

Best Case:Center Location(x,z) = 13.590499999999999 mm and 38.4538 mm , Average Metric1 per bubble: 0.274615, Average Metric2 per bubble: : 0.004607

** Experiment 5: **learning rate=1e-5, sigma of kernel f=2, regularizer parameter=0.01

**Percentage gap between training and validation error** ( \( \frac{validation errror – training error}{training error}*100 \) ) = 4.6

**Average quantitative metric1** is 12.216727

**Average metric1 per bubble** is 0.482225

**Average quantitative metric2** is 0.517551

**Average metric2 per bubble** is 0.020331

Worst Case:Center Location(x,z) = 29.6065 mm and 48.802 mm , Average Metric1 per bubble: 0.876782, Average Metric2 per bubble: 0.058363

Best Case:Center Location(x,z) = 13.590499999999999 mm and 38.4538 mm , Average Metric1 per bubble: 0.378525, Average Metric2 per bubble: : 0.012077

** Experiment 6: **learning rate=1e-5, sigma of kernel f=2, regularizer parameter=0.005

**Percentage gap between training and validation error** ( \( \frac{validation errror – training error}{training error}*100 \) ) = 9.2

**Average quantitative metric1** is 11.252816

**Average metric1 per bubble** is 0.444525

**Average quantitative metric2** is 0.393360

**Average metric2 per bubble** is 0.015460

Worst Case:Center Location(x,z) = 29.6065 mm and 48.8026 mm , Average Metric1 per bubble: 0.941545, Average Metric2 per bubble: : 0.057581

Best Case:Center Location(x,z) = 14.8225 mm and 37.961 mm , Average Metric1 per bubble: 0.335283, Average Metric2 per bubble: : 0.008589