|
Model pushed to 1 GPU(s), type NVIDIA GeForce RTX 3090. |
|
Loaded pretrained model from /data/shared/luojun/runs/snow1/weights/checkpoint.pth.tar |
|
--- Training --- |
|
Epoch 0 train loss: 0.6085, acc: 0.8926 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.829 |
|
mIoU improved from 0.0000 to 0.4715. |
|
mIoU improved from 0.0000 to 0.6457. |
|
--- Training --- |
|
Epoch 1 train loss: 0.5992, acc: 0.8947 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.854 |
|
mIoU improved from 0.4715 to 0.4840. |
|
mIoU improved from 0.6457 to 0.6482. |
|
--- Training --- |
|
Epoch 2 train loss: 0.5943, acc: 0.8953 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.839 |
|
--- Training --- |
|
Epoch 3 train loss: 0.5837, acc: 0.8977 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.828 |
|
--- Training --- |
|
Epoch 4 train loss: 0.5794, acc: 0.8984 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.847 |
|
mIoU improved from 0.6482 to 0.6508. |
|
--- Training --- |
|
Epoch 5 train loss: 0.5752, acc: 0.8990 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.816 |
|
--- Training --- |
|
Epoch 6 train loss: 0.5719, acc: 0.8999 |
|
--- Validation - Source --- |
|
* Acc 0.932 |
|
--- Validation - Target --- |
|
* Acc 0.833 |
|
--- Training --- |
|
Epoch 7 train loss: 0.5682, acc: 0.9004 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.843 |
|
--- Training --- |
|
Epoch 8 train loss: 0.5659, acc: 0.9010 |
|
--- Validation - Source --- |
|
* Acc 0.931 |
|
--- Validation - Target --- |
|
* Acc 0.847 |
|
--- Training --- |
|
Epoch 9 train loss: 0.5675, acc: 0.9005 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.826 |
|
--- Training --- |
|
Epoch 10 train loss: 0.5579, acc: 0.9027 |
|
--- Validation - Source --- |
|
* Acc 0.932 |
|
--- Validation - Target --- |
|
* Acc 0.838 |
|
--- Training --- |
|
Epoch 11 train loss: 0.5569, acc: 0.9024 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.836 |
|
--- Training --- |
|
Epoch 12 train loss: 0.5530, acc: 0.9033 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.832 |
|
--- Training --- |
|
Epoch 13 train loss: 0.5535, acc: 0.9032 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.857 |
|
--- Training --- |
|
Epoch 14 train loss: 0.5509, acc: 0.9035 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.825 |
|
--- Training --- |
|
Epoch 15 train loss: 0.5475, acc: 0.9043 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.835 |
|
--- Training --- |
|
Epoch 16 train loss: 0.5471, acc: 0.9041 |
|
--- Validation - Source --- |
|
* Acc 0.932 |
|
--- Validation - Target --- |
|
* Acc 0.820 |
|
--- Training --- |
|
Epoch 17 train loss: 0.5439, acc: 0.9052 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.833 |
|
--- Training --- |
|
Epoch 18 train loss: 0.5415, acc: 0.9053 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.845 |
|
--- Training --- |
|
Epoch 19 train loss: 0.5382, acc: 0.9059 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.837 |
|
mIoU improved from 0.6508 to 0.6520. |
|
--- Training --- |
|
Epoch 20 train loss: 0.5375, acc: 0.9062 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.831 |
|
mIoU improved from 0.6520 to 0.6532. |
|
--- Training --- |
|
Epoch 21 train loss: 0.5339, acc: 0.9067 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.835 |
|
mIoU improved from 0.6532 to 0.6549. |
|
--- Training --- |
|
Epoch 22 train loss: 0.5356, acc: 0.9058 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.818 |
|
--- Training --- |
|
Epoch 23 train loss: 0.5318, acc: 0.9071 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.841 |
|
--- Training --- |
|
Epoch 24 train loss: 0.5310, acc: 0.9072 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.830 |
|
--- Training --- |
|
Epoch 25 train loss: 0.5306, acc: 0.9070 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.834 |
|
--- Training --- |
|
Epoch 26 train loss: 0.5282, acc: 0.9079 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.836 |
|
--- Training --- |
|
Epoch 27 train loss: 0.5270, acc: 0.9080 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.848 |
|
--- Training --- |
|
Epoch 28 train loss: 0.5243, acc: 0.9084 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.848 |
|
--- Training --- |
|
Epoch 29 train loss: 0.5216, acc: 0.9087 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.841 |
|
--- Training --- |
|
Epoch 30 train loss: 0.5198, acc: 0.9094 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.847 |
|
--- Training --- |
|
Epoch 31 train loss: 0.5196, acc: 0.9094 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.860 |
|
--- Training --- |
|
Epoch 32 train loss: 0.5171, acc: 0.9096 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.840 |
|
--- Training --- |
|
Epoch 33 train loss: 0.5165, acc: 0.9098 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.840 |
|
--- Training --- |
|
Epoch 34 train loss: 0.5156, acc: 0.9099 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.826 |
|
mIoU improved from 0.6549 to 0.6551. |
|
--- Training --- |
|
Epoch 35 train loss: 0.5144, acc: 0.9103 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.838 |
|
--- Training --- |
|
Epoch 36 train loss: 0.5123, acc: 0.9107 |
|
--- Validation - Source --- |
|
* Acc 0.933 |
|
--- Validation - Target --- |
|
* Acc 0.832 |
|
--- Training --- |
|
Epoch 37 train loss: 0.5160, acc: 0.9102 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.833 |
|
mIoU improved from 0.6551 to 0.6556. |
|
--- Training --- |
|
Epoch 38 train loss: 0.5113, acc: 0.9106 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.824 |
|
--- Training --- |
|
Epoch 39 train loss: 0.5128, acc: 0.9100 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.826 |
|
--- Training --- |
|
Epoch 40 train loss: 0.5097, acc: 0.9114 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.847 |
|
--- Training --- |
|
Epoch 41 train loss: 0.5076, acc: 0.9112 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.839 |
|
--- Training --- |
|
Epoch 42 train loss: 0.5074, acc: 0.9114 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.842 |
|
mIoU improved from 0.6556 to 0.6569. |
|
--- Training --- |
|
Epoch 43 train loss: 0.5054, acc: 0.9119 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.840 |
|
mIoU improved from 0.6569 to 0.6578. |
|
--- Training --- |
|
Epoch 44 train loss: 0.5049, acc: 0.9118 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.838 |
|
--- Training --- |
|
Epoch 45 train loss: 0.5026, acc: 0.9125 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.837 |
|
--- Training --- |
|
Epoch 46 train loss: 0.5046, acc: 0.9120 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.839 |
|
mIoU improved from 0.6578 to 0.6579. |
|
--- Training --- |
|
Epoch 47 train loss: 0.5023, acc: 0.9125 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.822 |
|
--- Training --- |
|
Epoch 48 train loss: 0.4998, acc: 0.9128 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.826 |
|
mIoU improved from 0.6579 to 0.6581. |
|
--- Training --- |
|
Epoch 49 train loss: 0.4978, acc: 0.9131 |
|
--- Validation - Source --- |
|
* Acc 0.934 |
|
--- Validation - Target --- |
|
* Acc 0.805 |
|
--- Training --- |
|
Epoch 50 train loss: 0.4982, acc: 0.9132 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.793 |
|
--- Training --- |
|
Epoch 51 train loss: 0.4959, acc: 0.9136 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.839 |
|
--- Training --- |
|
Epoch 52 train loss: 0.4948, acc: 0.9138 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.828 |
|
--- Training --- |
|
Epoch 53 train loss: 0.4965, acc: 0.9132 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.824 |
|
--- Training --- |
|
Epoch 54 train loss: 0.4954, acc: 0.9134 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.849 |
|
mIoU improved from 0.6581 to 0.6589. |
|
--- Training --- |
|
Epoch 55 train loss: 0.4943, acc: 0.9138 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.857 |
|
--- Training --- |
|
Epoch 56 train loss: 0.4943, acc: 0.9137 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.819 |
|
--- Training --- |
|
Epoch 57 train loss: 0.4944, acc: 0.9137 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.845 |
|
--- Training --- |
|
Epoch 58 train loss: 0.4924, acc: 0.9141 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.837 |
|
--- Training --- |
|
Epoch 59 train loss: 0.4903, acc: 0.9143 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.823 |
|
--- Training --- |
|
Epoch 60 train loss: 0.4906, acc: 0.9139 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.847 |
|
--- Training --- |
|
Epoch 61 train loss: 0.4900, acc: 0.9145 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.834 |
|
--- Training --- |
|
Epoch 62 train loss: 0.4918, acc: 0.9141 |
|
--- Validation - Source --- |
|
* Acc 0.937 |
|
--- Validation - Target --- |
|
* Acc 0.847 |
|
mIoU improved from 0.6589 to 0.6596. |
|
--- Training --- |
|
Epoch 63 train loss: 0.4899, acc: 0.9145 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.841 |
|
--- Training --- |
|
Epoch 64 train loss: 0.4886, acc: 0.9148 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.836 |
|
--- Training --- |
|
Epoch 65 train loss: 0.4874, acc: 0.9151 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.840 |
|
--- Training --- |
|
Epoch 66 train loss: 0.4842, acc: 0.9153 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.833 |
|
--- Training --- |
|
Epoch 67 train loss: 0.4847, acc: 0.9153 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.850 |
|
--- Training --- |
|
Epoch 68 train loss: 0.4836, acc: 0.9158 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.849 |
|
mIoU improved from 0.6596 to 0.6599. |
|
--- Training --- |
|
Epoch 69 train loss: 0.4842, acc: 0.9156 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.835 |
|
mIoU improved from 0.6599 to 0.6629. |
|
--- Training --- |
|
Epoch 70 train loss: 0.4835, acc: 0.9158 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.848 |
|
--- Training --- |
|
Epoch 71 train loss: 0.4861, acc: 0.9152 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.845 |
|
--- Training --- |
|
Epoch 72 train loss: 0.4815, acc: 0.9159 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.839 |
|
--- Training --- |
|
Epoch 73 train loss: 0.4819, acc: 0.9159 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.826 |
|
--- Training --- |
|
Epoch 74 train loss: 0.4832, acc: 0.9157 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.844 |
|
--- Training --- |
|
Epoch 75 train loss: 0.4829, acc: 0.9160 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.845 |
|
--- Training --- |
|
Epoch 76 train loss: 0.4840, acc: 0.9157 |
|
--- Validation - Source --- |
|
* Acc 0.935 |
|
--- Validation - Target --- |
|
* Acc 0.833 |
|
--- Training --- |
|
Epoch 77 train loss: 0.4781, acc: 0.9165 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.825 |
|
--- Training --- |
|
Epoch 78 train loss: 0.4819, acc: 0.9158 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.809 |
|
--- Training --- |
|
Epoch 79 train loss: 0.4801, acc: 0.9163 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.834 |
|
--- Training --- |
|
Epoch 80 train loss: 0.4822, acc: 0.9161 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.851 |
|
--- Training --- |
|
Epoch 81 train loss: 0.4770, acc: 0.9166 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.822 |
|
--- Training --- |
|
Epoch 82 train loss: 0.4809, acc: 0.9163 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.830 |
|
--- Training --- |
|
Epoch 83 train loss: 0.4776, acc: 0.9166 |
|
--- Validation - Source --- |
|
* Acc 0.937 |
|
--- Validation - Target --- |
|
* Acc 0.815 |
|
mIoU improved from 0.6629 to 0.6652. |
|
--- Training --- |
|
Epoch 84 train loss: 0.4795, acc: 0.9163 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.833 |
|
--- Training --- |
|
Epoch 85 train loss: 0.4802, acc: 0.9163 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.851 |
|
--- Training --- |
|
Epoch 86 train loss: 0.4788, acc: 0.9162 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.847 |
|
--- Training --- |
|
Epoch 87 train loss: 0.4777, acc: 0.9167 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.832 |
|
--- Training --- |
|
Epoch 88 train loss: 0.4757, acc: 0.9173 |
|
--- Validation - Source --- |
|
* Acc 0.937 |
|
--- Validation - Target --- |
|
* Acc 0.832 |
|
--- Training --- |
|
Epoch 89 train loss: 0.4772, acc: 0.9169 |
|
--- Validation - Source --- |
|
* Acc 0.937 |
|
--- Validation - Target --- |
|
* Acc 0.838 |
|
--- Training --- |
|
Epoch 90 train loss: 0.4767, acc: 0.9169 |
|
--- Validation - Source --- |
|
* Acc 0.937 |
|
--- Validation - Target --- |
|
* Acc 0.823 |
|
--- Training --- |
|
Epoch 91 train loss: 0.4745, acc: 0.9174 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.832 |
|
--- Training --- |
|
Epoch 92 train loss: 0.4764, acc: 0.9171 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.844 |
|
--- Training --- |
|
Epoch 93 train loss: 0.4765, acc: 0.9168 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.842 |
|
--- Training --- |
|
Epoch 94 train loss: 0.4763, acc: 0.9169 |
|
--- Validation - Source --- |
|
* Acc 0.937 |
|
--- Validation - Target --- |
|
* Acc 0.841 |
|
--- Training --- |
|
Epoch 95 train loss: 0.4769, acc: 0.9170 |
|
--- Validation - Source --- |
|
* Acc 0.937 |
|
--- Validation - Target --- |
|
* Acc 0.821 |
|
--- Training --- |
|
Epoch 96 train loss: 0.4754, acc: 0.9170 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.840 |
|
--- Training --- |
|
Epoch 97 train loss: 0.4773, acc: 0.9169 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.831 |
|
--- Training --- |
|
Epoch 98 train loss: 0.4775, acc: 0.9168 |
|
--- Validation - Source --- |
|
* Acc 0.936 |
|
--- Validation - Target --- |
|
* Acc 0.840 |
|
--- Training --- |
|
Epoch 99 train loss: 0.4746, acc: 0.9171 |
|
--- Validation - Source --- |
|
* Acc 0.937 |
|
--- Validation - Target --- |
|
* Acc 0.840 |
|
Training complete in 333m 51s |
|
Best mIoU target: 0.484036 |
|
|