Is there something wrong?

[litingsjj]

Hi, thanks for your work! I have a question about grafting. I run grafting.sh, the accuracy is 93.910. And the result is 93.430, when train without grafting. It seems normal. but I calculate the number of invalid filters. The invalid filters ratio is 0.0366041362285614 without grafting, the result is 0.0404568612575531 with grafting.

[litingsjj]

Also, I train mobilenetv2 in cifar-10. here is the acc with grafting:
Namespace(a=0.4, batch_size=128, c=500, cifar=10, cos=True, device='cuda', difflr=True, epochs=200, i=1, lr=0.1, model='mobilenetv2', num=2, print_frequence=1000, r=None, s='checkpoint/grafting_cifar10_mobilenetv2')
Session:checkpoint/grafting_cifar10_mobilenetv2 Model:1 PID:27213
Files already downloaded and verified
Files already downloaded and verified
Network:1 epoch:0 accuracy:57.840 best:57.840
Network:1 epoch:1 accuracy:66.350 best:66.350
Network:1 epoch:2 accuracy:65.930 best:65.930
Network:1 epoch:3 accuracy:77.170 best:77.170
Network:1 epoch:4 accuracy:78.120 best:78.120
Network:1 epoch:5 accuracy:77.580 best:77.580
Network:1 epoch:6 accuracy:78.240 best:78.240
Network:1 epoch:7 accuracy:79.590 best:79.590
Network:1 epoch:8 accuracy:81.700 best:81.700
Network:1 epoch:9 accuracy:83.650 best:83.650
Network:1 epoch:10 accuracy:82.610 best:82.610
Network:1 epoch:11 accuracy:80.920 best:80.920
Network:1 epoch:12 accuracy:83.890 best:83.890
Network:1 epoch:13 accuracy:80.660 best:80.660
Network:1 epoch:14 accuracy:83.020 best:83.020
Network:1 epoch:15 accuracy:84.920 best:84.920
Network:1 epoch:16 accuracy:84.610 best:84.610
Network:1 epoch:17 accuracy:82.840 best:82.840
Network:1 epoch:18 accuracy:83.060 best:83.060
Network:1 epoch:19 accuracy:85.130 best:85.130
Network:1 epoch:20 accuracy:83.520 best:83.520
Network:1 epoch:21 accuracy:85.230 best:85.230
Network:1 epoch:22 accuracy:83.400 best:83.400
Network:1 epoch:23 accuracy:84.080 best:84.080
Network:1 epoch:24 accuracy:84.130 best:84.130
Network:1 epoch:25 accuracy:85.510 best:85.510
Network:1 epoch:26 accuracy:84.990 best:84.990
Network:1 epoch:27 accuracy:84.680 best:84.680
Network:1 epoch:28 accuracy:84.500 best:84.500
Network:1 epoch:29 accuracy:82.240 best:82.240
Network:1 epoch:30 accuracy:84.760 best:84.760
Network:1 epoch:31 accuracy:84.400 best:84.400
Network:1 epoch:32 accuracy:84.620 best:84.620
Network:1 epoch:33 accuracy:86.520 best:86.520
Network:1 epoch:34 accuracy:85.750 best:85.750
Network:1 epoch:35 accuracy:84.960 best:84.960
Network:1 epoch:36 accuracy:84.010 best:84.010
Network:1 epoch:37 accuracy:83.230 best:83.230
Network:1 epoch:38 accuracy:85.250 best:85.250
Network:1 epoch:39 accuracy:85.460 best:85.460
Network:1 epoch:40 accuracy:84.850 best:84.850
Network:1 epoch:41 accuracy:85.180 best:85.180
Network:1 epoch:42 accuracy:83.810 best:83.810
Network:1 epoch:43 accuracy:85.830 best:85.830
Network:1 epoch:44 accuracy:84.930 best:84.930
Network:1 epoch:45 accuracy:85.600 best:85.600
Network:1 epoch:46 accuracy:85.470 best:85.470
Network:1 epoch:47 accuracy:85.440 best:85.440
Network:1 epoch:48 accuracy:84.350 best:84.350
Network:1 epoch:49 accuracy:83.840 best:83.840
Network:1 epoch:50 accuracy:86.430 best:86.430
Network:1 epoch:51 accuracy:86.020 best:86.020
Network:1 epoch:52 accuracy:84.950 best:84.950
Network:1 epoch:53 accuracy:86.100 best:86.100
Network:1 epoch:54 accuracy:85.580 best:85.580
Network:1 epoch:55 accuracy:86.760 best:86.760
Network:1 epoch:56 accuracy:86.190 best:86.190
Network:1 epoch:57 accuracy:86.690 best:86.690
Network:1 epoch:58 accuracy:86.330 best:86.330
Network:1 epoch:59 accuracy:87.650 best:87.650
Network:1 epoch:60 accuracy:86.790 best:86.790
Network:1 epoch:61 accuracy:86.620 best:86.620
Network:1 epoch:62 accuracy:86.640 best:86.640
Network:1 epoch:63 accuracy:87.100 best:87.100
Network:1 epoch:64 accuracy:87.650 best:87.650
Network:1 epoch:65 accuracy:86.460 best:86.460
Network:1 epoch:66 accuracy:85.940 best:85.940
Network:1 epoch:67 accuracy:86.230 best:86.230
Network:1 epoch:68 accuracy:88.190 best:88.190
Network:1 epoch:69 accuracy:87.370 best:87.370
Network:1 epoch:70 accuracy:85.240 best:85.240
Network:1 epoch:71 accuracy:86.700 best:86.700
Network:1 epoch:72 accuracy:86.490 best:86.490
Network:1 epoch:73 accuracy:87.640 best:87.640
Network:1 epoch:74 accuracy:87.260 best:87.260
Network:1 epoch:75 accuracy:87.490 best:87.490
Network:1 epoch:76 accuracy:87.380 best:87.380
Network:1 epoch:77 accuracy:87.200 best:87.200
Network:1 epoch:78 accuracy:87.080 best:87.080
Network:1 epoch:79 accuracy:86.500 best:86.500
Network:1 epoch:80 accuracy:87.430 best:87.430
Network:1 epoch:81 accuracy:87.290 best:87.290
Network:1 epoch:82 accuracy:86.600 best:86.600
Network:1 epoch:83 accuracy:87.380 best:87.380
Network:1 epoch:84 accuracy:86.190 best:86.190
Network:1 epoch:85 accuracy:88.140 best:88.140
Network:1 epoch:86 accuracy:87.290 best:87.290
Network:1 epoch:87 accuracy:87.670 best:87.670
Network:1 epoch:88 accuracy:88.420 best:88.420
Network:1 epoch:89 accuracy:88.540 best:88.540
Network:1 epoch:90 accuracy:87.570 best:87.570
Network:1 epoch:91 accuracy:85.670 best:85.670
Network:1 epoch:92 accuracy:89.200 best:89.200
Network:1 epoch:93 accuracy:88.200 best:88.200
Network:1 epoch:94 accuracy:89.030 best:89.030
Network:1 epoch:95 accuracy:87.840 best:87.840
Network:1 epoch:96 accuracy:89.510 best:89.510
Network:1 epoch:97 accuracy:88.020 best:88.020
Network:1 epoch:98 accuracy:88.420 best:88.420
Network:1 epoch:99 accuracy:89.440 best:89.440
Network:1 epoch:100 accuracy:88.390 best:88.390
Network:1 epoch:101 accuracy:88.350 best:88.350
Network:1 epoch:102 accuracy:87.860 best:87.860
Network:1 epoch:103 accuracy:88.980 best:88.980
Network:1 epoch:104 accuracy:89.450 best:89.450
Network:1 epoch:105 accuracy:88.910 best:88.910
Network:1 epoch:106 accuracy:89.760 best:89.760
Network:1 epoch:107 accuracy:89.360 best:89.360
Network:1 epoch:108 accuracy:89.410 best:89.410
Network:1 epoch:109 accuracy:89.820 best:89.820
Network:1 epoch:110 accuracy:90.100 best:90.100
Network:1 epoch:111 accuracy:90.660 best:90.660
Network:1 epoch:112 accuracy:89.570 best:89.570
Network:1 epoch:113 accuracy:89.640 best:89.640
Network:1 epoch:114 accuracy:89.950 best:89.950
Network:1 epoch:115 accuracy:90.500 best:90.500
Network:1 epoch:116 accuracy:89.530 best:89.530
Network:1 epoch:117 accuracy:90.290 best:90.290
Network:1 epoch:118 accuracy:90.410 best:90.410
Network:1 epoch:119 accuracy:90.150 best:90.150
Network:1 epoch:120 accuracy:89.880 best:89.880
Network:1 epoch:121 accuracy:90.490 best:90.490
Network:1 epoch:122 accuracy:91.410 best:91.410
Network:1 epoch:123 accuracy:90.520 best:90.520
Network:1 epoch:124 accuracy:90.730 best:90.730
Network:1 epoch:125 accuracy:90.870 best:90.870
Network:1 epoch:126 accuracy:91.060 best:91.060
Network:1 epoch:127 accuracy:91.280 best:91.280
Network:1 epoch:128 accuracy:90.180 best:90.180
Network:1 epoch:129 accuracy:90.700 best:90.700
Network:1 epoch:130 accuracy:90.560 best:90.560
Network:1 epoch:131 accuracy:91.170 best:91.170
Network:1 epoch:132 accuracy:91.290 best:91.290
Network:1 epoch:133 accuracy:91.530 best:91.530
Network:1 epoch:134 accuracy:91.920 best:91.920
Network:1 epoch:135 accuracy:91.470 best:91.470
Network:1 epoch:136 accuracy:91.310 best:91.310
Network:1 epoch:137 accuracy:91.590 best:91.590
Network:1 epoch:138 accuracy:91.720 best:91.720
Network:1 epoch:139 accuracy:91.510 best:91.510
Network:1 epoch:140 accuracy:91.380 best:91.380
Network:1 epoch:141 accuracy:91.810 best:91.810
Network:1 epoch:142 accuracy:91.810 best:91.810
Network:1 epoch:143 accuracy:91.590 best:91.590
Network:1 epoch:144 accuracy:92.240 best:92.240
Network:1 epoch:145 accuracy:91.980 best:91.980
Network:1 epoch:146 accuracy:92.070 best:92.070
Network:1 epoch:147 accuracy:92.520 best:92.520
Network:1 epoch:148 accuracy:92.190 best:92.190
Network:1 epoch:149 accuracy:92.250 best:92.250
Network:1 epoch:150 accuracy:92.480 best:92.480
Network:1 epoch:151 accuracy:92.970 best:92.970
Network:1 epoch:152 accuracy:92.610 best:92.610
Network:1 epoch:153 accuracy:92.840 best:92.840
Network:1 epoch:154 accuracy:92.950 best:92.950
Network:1 epoch:155 accuracy:92.820 best:92.820
Network:1 epoch:156 accuracy:92.640 best:92.640
Network:1 epoch:157 accuracy:92.800 best:92.800
Network:1 epoch:158 accuracy:92.650 best:92.650
Network:1 epoch:159 accuracy:92.860 best:92.860
Network:1 epoch:160 accuracy:92.780 best:92.780
Network:1 epoch:161 accuracy:93.120 best:93.120
Network:1 epoch:162 accuracy:92.790 best:92.790
Network:1 epoch:163 accuracy:92.760 best:92.760
Network:1 epoch:164 accuracy:93.310 best:93.310
Network:1 epoch:165 accuracy:93.060 best:93.060
Network:1 epoch:166 accuracy:93.050 best:93.050
Network:1 epoch:167 accuracy:93.340 best:93.340
Network:1 epoch:168 accuracy:93.530 best:93.530
Network:1 epoch:169 accuracy:93.340 best:93.340
Network:1 epoch:170 accuracy:93.480 best:93.480
Network:1 epoch:171 accuracy:93.480 best:93.480
Network:1 epoch:172 accuracy:93.510 best:93.510
Network:1 epoch:173 accuracy:93.590 best:93.590
Network:1 epoch:174 accuracy:93.590 best:93.590
Network:1 epoch:175 accuracy:93.730 best:93.730
Network:1 epoch:176 accuracy:93.520 best:93.520
Network:1 epoch:177 accuracy:93.780 best:93.780
Network:1 epoch:178 accuracy:93.720 best:93.720
Network:1 epoch:179 accuracy:93.790 best:93.790
Network:1 epoch:180 accuracy:93.770 best:93.770
Network:1 epoch:181 accuracy:93.750 best:93.750
Network:1 epoch:182 accuracy:93.660 best:93.660
Network:1 epoch:183 accuracy:93.740 best:93.740
Network:1 epoch:184 accuracy:93.760 best:93.760
Network:1 epoch:185 accuracy:93.760 best:93.760
Network:1 epoch:186 accuracy:93.770 best:93.770
Network:1 epoch:187 accuracy:93.890 best:93.890
Network:1 epoch:188 accuracy:93.880 best:93.880
Network:1 epoch:189 accuracy:93.830 best:93.830
Network:1 epoch:190 accuracy:93.860 best:93.860
Network:1 epoch:191 accuracy:93.830 best:93.830
Network:1 epoch:192 accuracy:94.020 best:94.020
Network:1 epoch:193 accuracy:93.940 best:93.940
Network:1 epoch:194 accuracy:93.870 best:93.870
Network:1 epoch:195 accuracy:93.940 best:93.940
Network:1 epoch:196 accuracy:93.920 best:93.920
Network:1 epoch:197 accuracy:93.920 best:93.920
Network:1 epoch:198 accuracy:93.930 best:93.930
Network:1 epoch:199 accuracy:93.830 best:93.830

Namespace(a=0.4, batch_size=128, c=500, cifar=10, cos=True, device='cuda', difflr=True, epochs=200, i=2, lr=0.1, model='mobilenetv2', num=2, print_frequence=1000, r=None, s='checkpoint/grafting_cifar10_mobilenetv2')
Session:checkpoint/grafting_cifar10_mobilenetv2 Model:2 PID:27283
Files already downloaded and verified
Files already downloaded and verified
Network:2 epoch:0 accuracy:44.730 best:44.730
Network:2 epoch:1 accuracy:59.290 best:59.290
Network:2 epoch:2 accuracy:67.120 best:67.120
Network:2 epoch:3 accuracy:71.440 best:71.440
Network:2 epoch:4 accuracy:76.130 best:76.130
Network:2 epoch:5 accuracy:76.680 best:76.680
Network:2 epoch:6 accuracy:72.750 best:72.750
Network:2 epoch:7 accuracy:74.690 best:74.690
Network:2 epoch:8 accuracy:76.100 best:76.100
Network:2 epoch:9 accuracy:76.680 best:76.680
Network:2 epoch:10 accuracy:75.610 best:75.610
Network:2 epoch:11 accuracy:76.480 best:76.480
Network:2 epoch:12 accuracy:75.900 best:75.900
Network:2 epoch:13 accuracy:70.130 best:70.130
Network:2 epoch:14 accuracy:74.310 best:74.310
Network:2 epoch:15 accuracy:76.240 best:76.240
Network:2 epoch:16 accuracy:77.310 best:77.310
Network:2 epoch:17 accuracy:72.490 best:72.490
Network:2 epoch:18 accuracy:74.190 best:74.190
Network:2 epoch:19 accuracy:76.850 best:76.850
Network:2 epoch:20 accuracy:74.350 best:74.350
Network:2 epoch:21 accuracy:77.700 best:77.700
Network:2 epoch:22 accuracy:71.290 best:71.290
Network:2 epoch:23 accuracy:65.440 best:65.440
Network:2 epoch:24 accuracy:73.280 best:73.280
Network:2 epoch:25 accuracy:78.960 best:78.960
Network:2 epoch:26 accuracy:75.730 best:75.730
Network:2 epoch:27 accuracy:77.750 best:77.750
Network:2 epoch:28 accuracy:74.520 best:74.520
Network:2 epoch:29 accuracy:70.640 best:70.640
Network:2 epoch:30 accuracy:75.390 best:75.390
Network:2 epoch:31 accuracy:75.040 best:75.040
Network:2 epoch:32 accuracy:75.350 best:75.350
Network:2 epoch:33 accuracy:79.000 best:79.000
Network:2 epoch:34 accuracy:76.700 best:76.700
Network:2 epoch:35 accuracy:78.350 best:78.350
Network:2 epoch:36 accuracy:76.470 best:76.470
Network:2 epoch:37 accuracy:74.350 best:74.350
Network:2 epoch:38 accuracy:75.500 best:75.500
Network:2 epoch:39 accuracy:75.250 best:75.250
Network:2 epoch:40 accuracy:74.250 best:74.250
Network:2 epoch:41 accuracy:77.330 best:77.330
Network:2 epoch:42 accuracy:75.870 best:75.870
Network:2 epoch:43 accuracy:79.350 best:79.350
Network:2 epoch:44 accuracy:76.330 best:76.330
Network:2 epoch:45 accuracy:78.830 best:78.830
Network:2 epoch:46 accuracy:78.480 best:78.480
Network:2 epoch:47 accuracy:75.260 best:75.260
Network:2 epoch:48 accuracy:81.100 best:81.100
Network:2 epoch:49 accuracy:76.380 best:76.380
Network:2 epoch:50 accuracy:71.610 best:71.610
Network:2 epoch:51 accuracy:75.360 best:75.360
Network:2 epoch:52 accuracy:80.710 best:80.710
Network:2 epoch:53 accuracy:77.650 best:77.650
Network:2 epoch:54 accuracy:73.040 best:73.040
Network:2 epoch:55 accuracy:78.590 best:78.590
Network:2 epoch:56 accuracy:75.000 best:75.000
Network:2 epoch:57 accuracy:77.640 best:77.640
Network:2 epoch:58 accuracy:76.930 best:76.930
Network:2 epoch:59 accuracy:78.720 best:78.720
Network:2 epoch:60 accuracy:76.490 best:76.490
Network:2 epoch:61 accuracy:79.290 best:79.290
Network:2 epoch:62 accuracy:79.310 best:79.310
Network:2 epoch:63 accuracy:80.010 best:80.010
Network:2 epoch:64 accuracy:82.010 best:82.010
Network:2 epoch:65 accuracy:78.190 best:78.190
Network:2 epoch:66 accuracy:78.040 best:78.040
Network:2 epoch:67 accuracy:79.530 best:79.530
Network:2 epoch:68 accuracy:79.920 best:79.920
Network:2 epoch:69 accuracy:72.750 best:72.750
Network:2 epoch:70 accuracy:78.140 best:78.140
Network:2 epoch:71 accuracy:79.970 best:79.970
Network:2 epoch:72 accuracy:78.940 best:78.940
Network:2 epoch:73 accuracy:76.960 best:76.960
Network:2 epoch:74 accuracy:79.890 best:79.890
Network:2 epoch:75 accuracy:81.280 best:81.280
Network:2 epoch:76 accuracy:79.650 best:79.650
Network:2 epoch:77 accuracy:78.090 best:78.090
Network:2 epoch:78 accuracy:82.450 best:82.450
Network:2 epoch:79 accuracy:80.330 best:80.330
Network:2 epoch:80 accuracy:75.760 best:75.760
Network:2 epoch:81 accuracy:77.230 best:77.230
Network:2 epoch:82 accuracy:81.770 best:81.770
Network:2 epoch:83 accuracy:80.520 best:80.520
Network:2 epoch:84 accuracy:79.710 best:79.710
Network:2 epoch:85 accuracy:81.530 best:81.530
Network:2 epoch:86 accuracy:78.380 best:78.380
Network:2 epoch:87 accuracy:80.630 best:80.630
Network:2 epoch:88 accuracy:80.390 best:80.390
Network:2 epoch:89 accuracy:82.360 best:82.360
Network:2 epoch:90 accuracy:83.250 best:83.250
Network:2 epoch:91 accuracy:79.590 best:79.590
Network:2 epoch:92 accuracy:83.000 best:83.000
Network:2 epoch:93 accuracy:82.150 best:82.150
Network:2 epoch:94 accuracy:83.680 best:83.680
Network:2 epoch:95 accuracy:82.030 best:82.030
Network:2 epoch:96 accuracy:81.700 best:81.700
Network:2 epoch:97 accuracy:79.650 best:79.650
Network:2 epoch:98 accuracy:83.200 best:83.200
Network:2 epoch:99 accuracy:81.930 best:81.930
Network:2 epoch:100 accuracy:80.480 best:80.480
Network:2 epoch:101 accuracy:83.930 best:83.930
Network:2 epoch:102 accuracy:83.230 best:83.230
Network:2 epoch:103 accuracy:83.900 best:83.900
Network:2 epoch:104 accuracy:83.630 best:83.630
Network:2 epoch:105 accuracy:84.260 best:84.260
Network:2 epoch:106 accuracy:82.040 best:82.040
Network:2 epoch:107 accuracy:82.550 best:82.550
Network:2 epoch:108 accuracy:84.330 best:84.330
Network:2 epoch:109 accuracy:82.230 best:82.230
Network:2 epoch:110 accuracy:84.860 best:84.860
Network:2 epoch:111 accuracy:82.340 best:82.340
Network:2 epoch:112 accuracy:85.620 best:85.620
Network:2 epoch:113 accuracy:84.680 best:84.680
Network:2 epoch:114 accuracy:84.710 best:84.710
Network:2 epoch:115 accuracy:84.430 best:84.430
Network:2 epoch:116 accuracy:82.650 best:82.650
Network:2 epoch:117 accuracy:86.010 best:86.010
Network:2 epoch:118 accuracy:82.720 best:82.720
Network:2 epoch:119 accuracy:82.460 best:82.460
Network:2 epoch:120 accuracy:84.180 best:84.180
Network:2 epoch:121 accuracy:83.390 best:83.390
Network:2 epoch:122 accuracy:85.700 best:85.700
Network:2 epoch:123 accuracy:85.990 best:85.990
Network:2 epoch:124 accuracy:86.200 best:86.200
Network:2 epoch:125 accuracy:86.710 best:86.710
Network:2 epoch:126 accuracy:85.840 best:85.840
Network:2 epoch:127 accuracy:85.300 best:85.300
Network:2 epoch:128 accuracy:88.210 best:88.210
Network:2 epoch:129 accuracy:85.770 best:85.770
Network:2 epoch:130 accuracy:87.280 best:87.280
Network:2 epoch:131 accuracy:86.470 best:86.470
Network:2 epoch:132 accuracy:87.010 best:87.010
Network:2 epoch:133 accuracy:85.740 best:85.740
Network:2 epoch:134 accuracy:88.300 best:88.300
Network:2 epoch:135 accuracy:86.170 best:86.170
Network:2 epoch:136 accuracy:87.590 best:87.590
Network:2 epoch:137 accuracy:88.220 best:88.220
Network:2 epoch:138 accuracy:87.930 best:87.930
Network:2 epoch:139 accuracy:89.020 best:89.020
Network:2 epoch:140 accuracy:88.230 best:88.230
Network:2 epoch:141 accuracy:88.220 best:88.220
Network:2 epoch:142 accuracy:88.110 best:88.110
Network:2 epoch:143 accuracy:88.280 best:88.280
Network:2 epoch:144 accuracy:86.850 best:86.850
Network:2 epoch:145 accuracy:89.250 best:89.250
Network:2 epoch:146 accuracy:89.240 best:89.240
Network:2 epoch:147 accuracy:88.850 best:88.850
Network:2 epoch:148 accuracy:87.470 best:87.470
Network:2 epoch:149 accuracy:89.910 best:89.910
Network:2 epoch:150 accuracy:88.710 best:88.710
Network:2 epoch:151 accuracy:88.710 best:88.710
Network:2 epoch:152 accuracy:90.190 best:90.190
Network:2 epoch:153 accuracy:90.240 best:90.240
Network:2 epoch:154 accuracy:90.010 best:90.010
Network:2 epoch:155 accuracy:90.080 best:90.080
Network:2 epoch:156 accuracy:89.520 best:89.520
Network:2 epoch:157 accuracy:89.970 best:89.970
Network:2 epoch:158 accuracy:90.860 best:90.860
Network:2 epoch:159 accuracy:89.660 best:89.660
Network:2 epoch:160 accuracy:89.800 best:89.800
Network:2 epoch:161 accuracy:90.410 best:90.410
Network:2 epoch:162 accuracy:91.150 best:91.150
Network:2 epoch:163 accuracy:91.520 best:91.520
Network:2 epoch:164 accuracy:91.180 best:91.180
Network:2 epoch:165 accuracy:91.550 best:91.550
Network:2 epoch:166 accuracy:91.140 best:91.140
Network:2 epoch:167 accuracy:92.150 best:92.150
Network:2 epoch:168 accuracy:91.800 best:91.800
Network:2 epoch:169 accuracy:91.900 best:91.900
Network:2 epoch:170 accuracy:92.750 best:92.750
Network:2 epoch:171 accuracy:92.640 best:92.640
Network:2 epoch:172 accuracy:92.740 best:92.740
Network:2 epoch:173 accuracy:92.920 best:92.920
Network:2 epoch:174 accuracy:93.050 best:93.050
Network:2 epoch:175 accuracy:92.960 best:92.960
Network:2 epoch:176 accuracy:93.340 best:93.340
Network:2 epoch:177 accuracy:93.490 best:93.490
Network:2 epoch:178 accuracy:93.510 best:93.510
Network:2 epoch:179 accuracy:93.520 best:93.520
Network:2 epoch:180 accuracy:93.390 best:93.390
Network:2 epoch:181 accuracy:93.650 best:93.650
Network:2 epoch:182 accuracy:93.510 best:93.510
Network:2 epoch:183 accuracy:93.520 best:93.520
Network:2 epoch:184 accuracy:93.780 best:93.780
Network:2 epoch:185 accuracy:93.640 best:93.640
Network:2 epoch:186 accuracy:93.750 best:93.750
Network:2 epoch:187 accuracy:93.660 best:93.660
Network:2 epoch:188 accuracy:93.760 best:93.760
Network:2 epoch:189 accuracy:93.840 best:93.840
Network:2 epoch:190 accuracy:93.950 best:93.950
Network:2 epoch:191 accuracy:93.930 best:93.930
Network:2 epoch:192 accuracy:93.830 best:93.830
Network:2 epoch:193 accuracy:93.920 best:93.920
Network:2 epoch:194 accuracy:93.940 best:93.940
Network:2 epoch:195 accuracy:93.870 best:93.870
Network:2 epoch:196 accuracy:93.760 best:93.760
Network:2 epoch:197 accuracy:93.910 best:93.910
Network:2 epoch:198 accuracy:93.860 best:93.860
Network:2 epoch:199 accuracy:93.800 best:93.800

without grafting as follow:
Namespace(a=0.4, batch_size=128, c=500, cifar=10, cos=True, device='cuda', difflr=True, epochs=200, i=3, lr=0.1, model='mobilenetv2', num=2, print_frequence=1000, r=None, s='checkpoint/grafting_cifar10_mobilenetv2')
Session:checkpoint/grafting_cifar10_mobilenetv2 Model:3 PID:29845
Files already downloaded and verified
Files already downloaded and verified
Network:3 epoch:0 accuracy:55.050 best:55.050
Network:3 epoch:1 accuracy:67.030 best:67.030
Network:3 epoch:2 accuracy:71.000 best:71.000
Network:3 epoch:3 accuracy:75.620 best:75.620
Network:3 epoch:4 accuracy:75.510 best:75.510
Network:3 epoch:5 accuracy:77.160 best:77.160
Network:3 epoch:6 accuracy:79.500 best:79.500
Network:3 epoch:7 accuracy:81.320 best:81.320
Network:3 epoch:8 accuracy:81.540 best:81.540
Network:3 epoch:9 accuracy:80.620 best:80.620
Network:3 epoch:10 accuracy:80.160 best:80.160
Network:3 epoch:11 accuracy:81.020 best:81.020
Network:3 epoch:12 accuracy:82.420 best:82.420
Network:3 epoch:13 accuracy:83.850 best:83.850
Network:3 epoch:14 accuracy:80.920 best:80.920
Network:3 epoch:15 accuracy:82.410 best:82.410
Network:3 epoch:16 accuracy:82.690 best:82.690
Network:3 epoch:17 accuracy:84.860 best:84.860
Network:3 epoch:18 accuracy:83.040 best:83.040
Network:3 epoch:19 accuracy:82.720 best:82.720
Network:3 epoch:20 accuracy:83.010 best:83.010
Network:3 epoch:21 accuracy:85.140 best:85.140
Network:3 epoch:22 accuracy:83.900 best:83.900
Network:3 epoch:23 accuracy:83.300 best:83.300
Network:3 epoch:24 accuracy:84.500 best:84.500
Network:3 epoch:25 accuracy:81.860 best:81.860
Network:3 epoch:26 accuracy:81.720 best:81.720
Network:3 epoch:27 accuracy:81.790 best:81.790
Network:3 epoch:28 accuracy:81.450 best:81.450
Network:3 epoch:29 accuracy:84.020 best:84.020
Network:3 epoch:30 accuracy:84.540 best:84.540
Network:3 epoch:31 accuracy:83.270 best:83.270
Network:3 epoch:32 accuracy:86.160 best:86.160
Network:3 epoch:33 accuracy:84.990 best:84.990
Network:3 epoch:34 accuracy:82.100 best:82.100
Network:3 epoch:35 accuracy:83.470 best:83.470
Network:3 epoch:36 accuracy:85.860 best:85.860
Network:3 epoch:37 accuracy:84.620 best:84.620
Network:3 epoch:38 accuracy:83.970 best:83.970
Network:3 epoch:39 accuracy:84.940 best:84.940
Network:3 epoch:40 accuracy:80.900 best:80.900
Network:3 epoch:41 accuracy:82.390 best:82.390
Network:3 epoch:42 accuracy:85.310 best:85.310
Network:3 epoch:43 accuracy:85.550 best:85.550
Network:3 epoch:44 accuracy:85.530 best:85.530
Network:3 epoch:45 accuracy:83.050 best:83.050
Network:3 epoch:46 accuracy:85.890 best:85.890
Network:3 epoch:47 accuracy:86.230 best:86.230
Network:3 epoch:48 accuracy:84.100 best:84.100
Network:3 epoch:49 accuracy:85.900 best:85.900
Network:3 epoch:50 accuracy:85.990 best:85.990
Network:3 epoch:51 accuracy:85.230 best:85.230
Network:3 epoch:52 accuracy:84.720 best:84.720
Network:3 epoch:53 accuracy:85.640 best:85.640
Network:3 epoch:54 accuracy:83.870 best:83.870
Network:3 epoch:55 accuracy:84.000 best:84.000
Network:3 epoch:56 accuracy:82.900 best:82.900
Network:3 epoch:57 accuracy:85.490 best:85.490
Network:3 epoch:58 accuracy:84.590 best:84.590
Network:3 epoch:59 accuracy:83.220 best:83.220
Network:3 epoch:60 accuracy:85.770 best:85.770
Network:3 epoch:61 accuracy:85.020 best:85.020
Network:3 epoch:62 accuracy:84.780 best:84.780
Network:3 epoch:63 accuracy:87.260 best:87.260
Network:3 epoch:64 accuracy:85.840 best:85.840
Network:3 epoch:65 accuracy:86.890 best:86.890
Network:3 epoch:66 accuracy:84.750 best:84.750
Network:3 epoch:67 accuracy:87.600 best:87.600
Network:3 epoch:68 accuracy:86.650 best:86.650
Network:3 epoch:69 accuracy:86.380 best:86.380
Network:3 epoch:70 accuracy:86.540 best:86.540
Network:3 epoch:71 accuracy:86.940 best:86.940
Network:3 epoch:72 accuracy:82.870 best:82.870
Network:3 epoch:73 accuracy:86.450 best:86.450
Network:3 epoch:74 accuracy:81.630 best:81.630
Network:3 epoch:75 accuracy:87.400 best:87.400
Network:3 epoch:76 accuracy:86.120 best:86.120
Network:3 epoch:77 accuracy:86.740 best:86.740
Network:3 epoch:78 accuracy:84.860 best:84.860
Network:3 epoch:79 accuracy:86.810 best:86.810
Network:3 epoch:80 accuracy:86.830 best:86.830
Network:3 epoch:81 accuracy:87.820 best:87.820
Network:3 epoch:82 accuracy:87.240 best:87.240
Network:3 epoch:83 accuracy:88.760 best:88.760
Network:3 epoch:84 accuracy:86.600 best:86.600
Network:3 epoch:85 accuracy:84.920 best:84.920
Network:3 epoch:86 accuracy:88.140 best:88.140
Network:3 epoch:87 accuracy:86.610 best:86.610
Network:3 epoch:88 accuracy:88.820 best:88.820
Network:3 epoch:89 accuracy:85.920 best:85.920
Network:3 epoch:90 accuracy:87.420 best:87.420
Network:3 epoch:91 accuracy:85.990 best:85.990
Network:3 epoch:92 accuracy:86.280 best:86.280
Network:3 epoch:93 accuracy:86.970 best:86.970
Network:3 epoch:94 accuracy:89.020 best:89.020
Network:3 epoch:95 accuracy:86.970 best:86.970
Network:3 epoch:96 accuracy:88.310 best:88.310
Network:3 epoch:97 accuracy:86.520 best:86.520
Network:3 epoch:98 accuracy:87.940 best:87.940
Network:3 epoch:99 accuracy:87.160 best:87.160
Network:3 epoch:100 accuracy:87.880 best:87.880
Network:3 epoch:101 accuracy:87.380 best:87.380
Network:3 epoch:102 accuracy:88.690 best:88.690
Network:3 epoch:103 accuracy:88.080 best:88.080
Network:3 epoch:104 accuracy:87.400 best:87.400
Network:3 epoch:105 accuracy:87.910 best:87.910
Network:3 epoch:106 accuracy:88.320 best:88.320
Network:3 epoch:107 accuracy:88.080 best:88.080
Network:3 epoch:108 accuracy:88.920 best:88.920
Network:3 epoch:109 accuracy:89.110 best:89.110
Network:3 epoch:110 accuracy:88.790 best:88.790
Network:3 epoch:111 accuracy:89.380 best:89.380
Network:3 epoch:112 accuracy:89.330 best:89.330
Network:3 epoch:113 accuracy:88.990 best:88.990
Network:3 epoch:114 accuracy:88.430 best:88.430
Network:3 epoch:115 accuracy:89.400 best:89.400
Network:3 epoch:116 accuracy:90.160 best:90.160
Network:3 epoch:117 accuracy:89.450 best:89.450
Network:3 epoch:118 accuracy:89.730 best:89.730
Network:3 epoch:119 accuracy:89.700 best:89.700
Network:3 epoch:120 accuracy:89.360 best:89.360
Network:3 epoch:121 accuracy:90.030 best:90.030
Network:3 epoch:122 accuracy:90.070 best:90.070
Network:3 epoch:123 accuracy:91.630 best:91.630
Network:3 epoch:124 accuracy:89.900 best:89.900
Network:3 epoch:125 accuracy:90.560 best:90.560
Network:3 epoch:126 accuracy:89.880 best:89.880
Network:3 epoch:127 accuracy:89.800 best:89.800
Network:3 epoch:128 accuracy:90.460 best:90.460
Network:3 epoch:129 accuracy:90.680 best:90.680
Network:3 epoch:130 accuracy:90.440 best:90.440
Network:3 epoch:131 accuracy:90.810 best:90.810
Network:3 epoch:132 accuracy:90.710 best:90.710
Network:3 epoch:133 accuracy:92.050 best:92.050
Network:3 epoch:134 accuracy:90.930 best:90.930
Network:3 epoch:135 accuracy:90.940 best:90.940
Network:3 epoch:136 accuracy:91.140 best:91.140
Network:3 epoch:137 accuracy:91.790 best:91.790
Network:3 epoch:138 accuracy:90.890 best:90.890
Network:3 epoch:139 accuracy:91.500 best:91.500
Network:3 epoch:140 accuracy:91.330 best:91.330
Network:3 epoch:141 accuracy:91.540 best:91.540
Network:3 epoch:142 accuracy:91.660 best:91.660
Network:3 epoch:143 accuracy:91.690 best:91.690
Network:3 epoch:144 accuracy:91.680 best:91.680
Network:3 epoch:145 accuracy:91.220 best:91.220
Network:3 epoch:146 accuracy:91.700 best:91.700
Network:3 epoch:147 accuracy:92.220 best:92.220
Network:3 epoch:148 accuracy:92.070 best:92.070
Network:3 epoch:149 accuracy:92.300 best:92.300
Network:3 epoch:150 accuracy:91.930 best:91.930
Network:3 epoch:151 accuracy:92.460 best:92.460
Network:3 epoch:152 accuracy:92.340 best:92.340
Network:3 epoch:153 accuracy:92.440 best:92.440
Network:3 epoch:154 accuracy:92.780 best:92.780
Network:3 epoch:155 accuracy:92.440 best:92.440
Network:3 epoch:156 accuracy:92.520 best:92.520
Network:3 epoch:157 accuracy:92.670 best:92.670
Network:3 epoch:158 accuracy:92.850 best:92.850
Network:3 epoch:159 accuracy:92.880 best:92.880
Network:3 epoch:160 accuracy:92.880 best:92.880
Network:3 epoch:161 accuracy:93.120 best:93.120
Network:3 epoch:162 accuracy:93.330 best:93.330
Network:3 epoch:163 accuracy:93.120 best:93.120
Network:3 epoch:164 accuracy:93.100 best:93.100
Network:3 epoch:165 accuracy:93.150 best:93.150
Network:3 epoch:166 accuracy:93.490 best:93.490
Network:3 epoch:167 accuracy:93.640 best:93.640
Network:3 epoch:168 accuracy:93.540 best:93.540
Network:3 epoch:169 accuracy:93.650 best:93.650
Network:3 epoch:170 accuracy:93.660 best:93.660
Network:3 epoch:171 accuracy:93.880 best:93.880
Network:3 epoch:172 accuracy:93.810 best:93.810
Network:3 epoch:173 accuracy:93.850 best:93.850
Network:3 epoch:174 accuracy:93.820 best:93.820
Network:3 epoch:175 accuracy:93.770 best:93.770
Network:3 epoch:176 accuracy:94.010 best:94.010
Network:3 epoch:177 accuracy:93.990 best:93.990
Network:3 epoch:178 accuracy:94.050 best:94.050
Network:3 epoch:179 accuracy:94.090 best:94.090
Network:3 epoch:180 accuracy:93.930 best:93.930
Network:3 epoch:181 accuracy:94.120 best:94.120
Network:3 epoch:182 accuracy:94.280 best:94.280
Network:3 epoch:183 accuracy:94.120 best:94.120
Network:3 epoch:184 accuracy:94.040 best:94.040
Network:3 epoch:185 accuracy:94.110 best:94.110
Network:3 epoch:186 accuracy:94.310 best:94.310
Network:3 epoch:187 accuracy:94.170 best:94.170
Network:3 epoch:188 accuracy:94.110 best:94.110
Network:3 epoch:189 accuracy:94.240 best:94.240
Network:3 epoch:190 accuracy:94.120 best:94.120
Network:3 epoch:191 accuracy:94.230 best:94.230
Network:3 epoch:192 accuracy:94.070 best:94.070
Network:3 epoch:193 accuracy:94.260 best:94.260
Network:3 epoch:194 accuracy:94.170 best:94.170
Network:3 epoch:195 accuracy:94.260 best:94.260
Network:3 epoch:196 accuracy:94.200 best:94.200
Network:3 epoch:197 accuracy:94.250 best:94.250
Network:3 epoch:198 accuracy:94.160 best:94.160
Network:3 epoch:199 accuracy:94.250 best:94.250

And the variable less than 0.001 in conv:
grafting:
total_nums: 2190784.0, total_filter_nums: 1238366.0, ratio: 0.5652615427970886
without grafting:
total_nums: 2190784.0, total_filter_nums: 978308.0, ratio: 0.44655612111091614
The invalid variable ratio is higher than without grafting???

[fxmeng]

Thank you for your interest in our work. We have a little update to our code.

1. we add the code for the baseline's result, named as baseline.py
2. We add the code for comparing the number of filters for the baseline model and the grafted model, named as filter_comparison.py

We notice that the number of filters you calculated does not match ours. I think you get a wrong definition of the `filter' in CNN. Please refer to the definition in https://blog.xrds.acm.org/2016/06/convolutional-neural-networks-cnns-illustrated-explanation.

For example, the number of filters in MobileNetV2 is not ‘2190784’ (you may calculate the number of two-dimensional slices of all the filters in CNN). In our calculation, the number of filters in MobileNetV2 is 10788 and the number of filters in ResNet32 is 1136.
Also for the test accuracy of MobileNetV2, we notice that in your implementation, the learning rate for baseline and grafting is different. We keep the entirely hyper-parameters to be the same for baseline and grafting (without modifying hyper-parameters to increase the diversity of grafting) and find grafting is better than baseline (See GitHub page). You may also run baseline.py and grafting.py to verify the results.

We also list the new results corresponding to the changes of invalid filters in our updated GitHub page. Please have a check. you may run filter_comparison.py to verify the result. Thank you for your interest in our work again.

[litingsjj]

I calculate the number of weights which less than 0.001, not filters.

[fxmeng]

Since the filter is the fundamental unit of the convolutional neural networks. We believe the calculation on filters, rather than weights, is more meaningful. Also, the whole paper concentrates on improving the 'filters' of the network. Thus the analysis of filters is more reasonable. We recommend you to refer filter_comparison.py for further verification.
Thanks.