TF-Agents 自定义环境 time_step 与 time_step_spec 不匹配的修复

在 TF-Agents 里自定义PyEnvironment时，_step和_reset单独调用都正常，一旦接上PyDriver和 Reverb 回放缓冲就开始报time_step and time_step_spec structures do not match。这个报错的字面表述容易把人带偏——它说的"结构"其实涵盖了 dtype、shape、bounds 和嵌套结构四类问题。问题现象环境模拟的是田径十项全能，内部状态用嵌套字典保存各项目的发展曲线参数。接入PyDriver后：File ".../tf_agents/drivers/py_driver.py", line 119, in run action_step = self.policy.action(time_step, policy_state) File ".../tf_agents/policies/tf_policy.py", line 317, in action nest_utils.assert_same_structure( File ".../tf_agents/utils/nest_utils.py", line 131, in assert_same_structure raise exception( ValueError: time_step and time_step_spec structures do not match