Skip to content

训练自己数据集的问题 #5

Description

@Zzhuoyuu

我使用我找到的数据集,由于它里面图片的大小和你的不一致(大部分大于256*256),所以我先压缩了大小,然后训练时出现了loss一直为nan的情况,请问你知道如何解决吗,感谢!

(pytorch) PS E:\QRCodeDetection-master\QRCodeDetection-master> python train.py -d ./resized_images -b 64 -e 80
2024-05-12 22:12:03.203769: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1666] function cupti_interface_->Subscribe( &subscriber_, (CUpti_CallbackFunc)ApiCallback, this)failed with error CUPTI could not be loaded or symbol could not be found.
2024-05-12 22:12:03.204291: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1757] function cupti_interface_->Finalize()failed with error CUPTI could not be loaded or symbol could not be found.
Epoch 1/80
E:\QRCodeDetection-master\QRCodeDetection-master\utils\util.py:134: RuntimeWarning: divide by zero encountered in log
tw = np.log(w / anchor_w)
E:\QRCodeDetection-master\QRCodeDetection-master\utils\util.py:135: RuntimeWarning: divide by zero encountered in log
th = np.log(h / anchor_h)
1/22 [>.............................] - ETA: 6:23 - loss: 29.2871 - score_acc: 0.5300 - cls_acc: 0.47972024-05-12 22:12:23.009745: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1666] function cupti_interface_->Subscribe( &subscriber_, (CUpti_CallbackFunc)ApiCallback, this)failed with error CUPTI could not be loaded or symbol could not be found.
2/22 [=>............................] - ETA: 4:12 - loss: 17.0141 - score_acc: 0.7628 - cls_acc: 0.39232024-05-12 22:12:34.730575: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1757] function cupti_interface_->Finalize()failed with error CUPTI could not be loaded or symbol could not be found.
22/22 [==============================] - 255s 11s/step - loss: nan - score_acc: 0.9753 - cls_acc: 0.4812
Epoch 2/80
22/22 [==============================] - 248s 11s/step - loss: nan - score_acc: 0.9966 - cls_acc: 0.9834
Epoch 3/80
22/22 [==============================] - 255s 11s/step - loss: nan - score_acc: 0.9966 - cls_acc: 0.9834
Epoch 4/80
22/22 [==============================] - 271s 12s/step - loss: nan - score_acc: 0.9966 - cls_acc: 0.9834
Epoch 5/80
22/22 [==============================] - 304s 14s/step - loss: nan - score_acc: 0.9966 - cls_acc: 0.9834 - val_loss: nan - val_score_acc: 0.9961 - val_cls_acc: 0.9774

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions