freqcodec自定义数据集训练问题

[Yakult-s]

[WJ-Server1] 2025-06-10 21:22:32,075 (gan_trainer:387) INFO: 36epoch:train:4201-4250batch:4250num_updates: iter_time=0.001, discriminator_forward_time=0.130, discriminator_total_loss=0.234, discriminator_loss=0.234, discriminator_backward_time=0.007, discriminator_optim_step_time=0.048, optim1_lr0=3.000e-04, discriminator_train_time=0.187, generator_forward_time=0.148, generator_loss=59.242, generator_recon_loss=0.010, generator_multi_spectral_recon_loss=0.447, generator_adv_loss=1.577, generator_feat_match_loss=0.120, generator_commit_loss=1.651, generator_enc_quant_loss=55.628, generator_backward_time=0.130, generator_optim_step_time=0.094, optim0_lr0=3.000e-04, generator_train_time=0.376, train_time=0.796
[WJ-Server1] 2025-06-10 21:23:12,392 (gan_trainer:387) INFO: 36epoch:train:4251-4300batch:4300num_updates: iter_time=9.480e-04, discriminator_forward_time=0.129, discriminator_total_loss=0.194, discriminator_loss=0.194, discriminator_backward_time=0.008, discriminator_optim_step_time=0.047, optim1_lr0=3.000e-04, discriminator_train_time=0.186, generator_forward_time=0.149, generator_loss=46.840, generator_recon_loss=0.010, generator_multi_spectral_recon_loss=0.450, generator_adv_loss=1.620, generator_feat_match_loss=0.125, generator_commit_loss=1.725, generator_enc_quant_loss=43.086, generator_backward_time=0.130, generator_optim_step_time=0.093, optim0_lr0=3.000e-04, generator_train_time=0.377, train_time=0.806
[WJ-Server1] 2025-06-10 21:23:52,834 (gan_trainer:387) INFO: 36epoch:train:4301-4350batch:4350num_updates: iter_time=0.001, discriminator_forward_time=0.129, discriminator_total_loss=0.294, discriminator_loss=0.294, discriminator_backward_time=0.008, discriminator_optim_step_time=0.047, optim1_lr0=3.000e-04, discriminator_train_time=0.186, generator_forward_time=0.148, generator_loss=55.556, generator_recon_loss=0.011, generator_multi_spectral_recon_loss=0.450, generator_adv_loss=1.712, generator_feat_match_loss=0.122, generator_commit_loss=1.447, generator_enc_quant_loss=52.102, generator_backward_time=0.130, generator_optim_step_time=0.093, optim0_lr0=3.000e-04, generator_train_time=0.376, train_time=0.809
[WJ-Server1] 2025-06-10 21:24:33,766 (gan_trainer:387) INFO: 36epoch:train:4351-4400batch:4400num_updates: iter_time=0.001, discriminator_forward_time=0.135, discriminator_total_loss=0.226, discriminator_loss=0.226, discriminator_backward_time=0.008, discriminator_optim_step_time=0.047, optim1_lr0=3.000e-04, discriminator_train_time=0.192, generator_forward_time=0.149, generator_loss=76.204, generator_recon_loss=0.011, generator_multi_spectral_recon_loss=0.481, generator_adv_loss=1.688, generator_feat_match_loss=0.126, generator_commit_loss=1.553, generator_enc_quant_loss=72.572, generator_backward_time=0.130, generator_optim_step_time=0.094, optim0_lr0=3.000e-04, generator_train_time=0.377, train_time=0.818
[WJ-Server1] 2025-06-10 21:25:13,781 (gan_trainer:387) INFO: 36epoch:train:4401-4450batch:4450num_updates: iter_time=6.183e-04, discriminator_forward_time=0.130, discriminator_total_loss=0.228, discriminator_loss=0.228, discriminator_backward_time=0.008, discriminator_optim_step_time=0.047, optim1_lr0=3.000e-04, discriminator_train_time=0.187, generator_forward_time=0.146, generator_loss=56.443, generator_recon_loss=0.010, generator_multi_spectral_recon_loss=0.436, generator_adv_loss=1.658, generator_feat_match_loss=0.118, generator_commit_loss=1.589, generator_enc_quant_loss=52.914, generator_backward_time=0.130, generator_optim_step_time=0.094, optim0_lr0=3.000e-04, generator_train_time=0.373, train_time=0.800
/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/kaldiio/utils.py:481: UserWarning: An error happend when loading "/mnt/disk_work/zp/data/VCTK_092_train_16k/p313/p313_421_mic2.wav"
warnings.warn('An error happend when loading "{}"'.format(ark_name))
[WJ-Server1] 2025-06-10 21:25:37,377 (dataset:414) ERROR: Error happened with path=./dump/foo/train/wav.scp, type=kaldi_ark, id=p313_421_mic2.wav
Traceback (most recent call last):
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/runpy.py", line 86, in run_code
exec(code, run_globals)
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/bin/codec_train.py", line 50, in
main(args=args)
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/bin/codec_train.py", line 23, in main
GANSpeechCodecTask.main(args=args, cmd=cmd)
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/tasks/abs_task.py", line 1130, in main
cls.main_worker(args)
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/tasks/abs_task.py", line 1431, in main_worker
cls.trainer.run(
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/train/trainer.py", line 309, in run
all_steps_are_invalid, max_update_stop = cls.train_one_epoch(
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/train/gan_trainer.py", line 154, in train_one_epoch
for iiter, (, batch_org) in enumerate(
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/train/reporter.py", line 274, in measure_iter_time
retval = next(iterator)
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 708, in next
data = self._next_data()
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1480, in _next_data
return self._process_data(data)
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1505, in _process_data
data.reraise()
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/torch/_utils.py", line 733, in reraise
raise exception
ValueError: Caught ValueError in DataLoader worker process 7.
Original Traceback (most recent call last):
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/kaldiio/matio.py", line 578, in read_ascii_mat
char = b.decode(encoding=default_encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 52, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/datasets/dataset.py", line 403, in getitem
value = loader[uid]
File "/home/zhoup241/git_clone/FunCodec-master/funcodec/datasets/dataset.py", line 56, in getitem
retval = self.loader[key]
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/kaldiio/utils.py", line 479, in getitem
return self._loader(ark_name)
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/kaldiio/matio.py", line 237, in load_mat
return _load_mat(fd, offset, slices, endian=endian)
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/kaldiio/matio.py", line 330, in _load_mat
array = read_kaldi(fd, endian)
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/kaldiio/matio.py", line 448, in read_kaldi
array = read_ascii_mat(fd)
File "/home/zhoup241/anaconda3/envs/pytorch1/lib/python3.10/site-packages/kaldiio/matio.py", line 580, in read_ascii_mat
raise ValueError("File format is wrong?")
ValueError: File format is wrong?

我在自定义数据集训练0.52M版本的frecodec时会出现以上错误，期间我更换了VCTK数据集和aishell3数据集,都在某个epoch出现了以上错误，我若把出问题的音频条目剔除掉后，后面某个wav音频文件还会报错，请问这种问题您遇到过吗？如何解决？