-
Notifications
You must be signed in to change notification settings - Fork 12
Description
When I was reproducing your project, I encountered a problem.
ray.rllib.utils.error.UnsupportedSpaceException: Action space (Box(-1.0, 1.0, (2,), float32)) of <ray.rllib.policy.tf_policy_template.HACO object at 0x000001D3693AF5C0> is not supported for SAC.
it seems that the SAC algorithm does not support action space(Ray version is 1.0.0 as required)
system:
windows 10
python3.7
ray1.0.0
tensorflow-gpu 2.3.1
error:
2025-07-27 15:44:02,814 ERROR trial_runner.py:567 -- Trial HACO_HumanInTheLoopEnv_68ba6_00000: Error processing event.
Traceback (most recent call last):
File "E:\conda\envs\haco\lib\site-packages\ray\tune\trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "E:\conda\envs\haco\lib\site-packages\ray\tune\ray_trial_executor.py", line 488, in fetch_result
result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
File "E:\conda\envs\haco\lib\site-packages\ray\worker.py", line 1428, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(UnsupportedSpaceException): ray::HACO.train() (pid=22984, ip=192.168.1.3)
File "python\ray_raylet.pyx", line 445, in ray._raylet.execute_task
File "E:\conda\envs\haco\lib\site-packages\ray\worker.py", line 174, in reraise_actor_init_error
raise self.actor_init_error
File "python\ray_raylet.pyx", line 479, in ray._raylet.execute_task
File "python\ray_raylet.pyx", line 483, in ray._raylet.execute_task
File "python\ray_raylet.pyx", line 484, in ray._raylet.execute_task
File "python\ray_raylet.pyx", line 438, in ray._raylet.execute_task.function_executor
File "E:\conda\envs\haco\lib\site-packages\ray\function_manager.py", line 553, in actor_method_executor
return method(actor, *args, **kwargs)
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\agents\trainer_template.py", line 101, in init
Trainer.init(self, config, env, logger_creator)
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\agents\trainer.py", line 476, in init
super().init(config, logger_creator)
File "E:\conda\envs\haco\lib\site-packages\ray\tune\trainable.py", line 249, in init
self.setup(copy.deepcopy(self.config))
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\agents\trainer.py", line 629, in setup
self._init(self.config, self.env_creator)
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\agents\trainer_template.py", line 125, in _init
self.config["num_workers"])
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\agents\trainer.py", line 699, in _make_workers
logdir=self.logdir)
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\evaluation\worker_set.py", line 74, in init
self._local_config)
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\evaluation\worker_set.py", line 305, in _make_worker
extra_python_environs=extra_python_environs)
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 416, in init
self._build_policy_map(policy_dict, policy_config)
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 1008, in _build_policy_map
policy_map[name] = cls(obs_space, act_space, merged_conf)
File "E:\conda\envs\haco\lib\site-packages\ray\rllib\policy\tf_policy_template.py", line 192, in init
validate_spaces(self, obs_space, action_space, config)
File "e:\env\hacotrain\haco\algo\sac_lag\sac_lag_policy.py", line 481, in validate_spaces
"SAC.".format(action_space, pid))
ray.rllib.utils.error.UnsupportedSpaceException: Action space (Box(-1.0, 1.0, (2,), float32)) of <ray.rllib.policy.tf_policy_template.HACO object at 0x000001D3693AF5C0> is not supported for SAC.