RayCaster and mesh func error

[wndml5140]

Hi, I’m currently trying to solve an indoor navigation task using reinforcement learning in Isaac Lab.
I ran into several issues related to LiDAR sensing and static obstacles, and I’m not sure how to fix them.

I imported my indoor wall geometry into Isaac Sim as a RigidObject.
However, when I attach a RayCaster LiDAR in Isaac Lab, the LiDAR rays do not detect the rigid object at all (all distances return inf).

So, I created a separate wall mesh for LiDAR sensing.
But when I spawn the mesh using UsdFileCfg().func(), the mesh does not stay in place — it jumps or flies upward during simulation, as if some unexpected transformation or physics is being applied.
Also, the raycaster does not detect wall mesh too.

At this point I’m not sure what the correct or recommended way is to spawn a static mesh object.

Also, in the Isaac Lab documentation, it says that RayCaster currently does not support multiple obstacles or dynamic obstacles very well.

Since I want to train an RL agent for indoor navigation with obstacle avoidance, I’m not sure what is the proper way to detect multiple static obstacles and moving obstacles.

Is there a recommended workflow for RL-based navigation with multiple obstacles?

@configclass
class OfficeDeliverEnvCfg(DirectRLEnvCfg):
    # env
    decimation = 4
    episode_length_s = 100

    sim: SimulationCfg = SimulationCfg(
        dt=1 / 60, 
        render_interval=decimation,
        device="cuda:0",  

    # robot(s)
    robot_cfg: ArticulationCfg = CARTER_CFG.replace(
        prim_path="/World/envs/env_.*/Robot",
        )
    
    scene: InteractiveSceneCfg = InteractiveSceneCfg(
        num_envs=1, 
        env_spacing=50.0, 
        replicate_physics=False,
        )

    wall_cfg = RigidObjectCfg(
        prim_path="/World/envs/env_.*/Wall",
        spawn=sim_utils.UsdFileCfg(
            usd_path=WALL_USD,
            scale=(1000.0, 1000.0, 1000.0),
            rigid_props=sim_utils.RigidBodyPropertiesCfg(
                disable_gravity=True,
                kinematic_enabled=True,
            ),
            collision_props=sim_utils.CollisionPropertiesCfg(
                contact_offset=0.02, rest_offset=0.0
            ),
        ),
    )


    wall_mesh_cfg = AssetBaseCfg(
        prim_path="/World/WallMesh",
        spawn=sim_utils.UsdFileCfg(
            usd_path=WALL_USD,     
            scale=(1000.0, 1000.0, 1000.0),
        ),
        # init_state=AssetBaseCfg.InitialStateCfg(
        #     pos=(0.0, 0.0, 0.0)
        # )
    )

    lidar_cfg: RayCasterCfg = RayCasterCfg( 
            prim_path="/World/envs/env_.*/Robot/Nova_Carter/chassis_link/sensors/XT_32/PandarXT_32_10hz",
            update_period=decimation * (1 / 60), 
            max_distance=20.0, 
            pattern_cfg=XT32_PATTERN_CFG, 
            debug_vis=False,
            

            mesh_prim_paths=["/World/WallMesh"],
        )

class OfficeDeliverEnv(DirectRLEnv):
    cfg: OfficeDeliverEnvCfg

    def __init__(self, cfg: OfficeDeliverEnvCfg, render_mode: str | None = None, **kwargs):
        super().__init__(cfg, render_mode, **kwargs)
        self._l_idx, _ = self.robot.find_joints("joint_wheel_left")
        self._r_idx, _ = self.robot.find_joints("joint_wheel_right")

        # self.joint_pos = self.robot.data.joint_pos
        # self.joint_vel = self.robot.data.joint_vel

        self.goal_pos = torch.tensor([20, 15, 0], device=self.device)

        num_envs = self.num_envs
        self._dist = torch.empty(num_envs, device=self.device)
        self._prev_dist = torch.empty(num_envs, device=self.device)
        self._heading_err = torch.empty(num_envs, device=self.device)
        self._prev_heading_err = torch.empty(num_envs, device=self.device)
        self._vel = torch.zeros(num_envs, device=self.device)
        self._ang_vel = torch.zeros(num_envs, device=self.device)

    def _setup_scene(self):
        # add ground plane
        # spawn_ground_plane(prim_path="/World/ground", cfg=GroundPlaneCfg())
        ground_cfg = sim_utils.GroundPlaneCfg()
        ground_cfg.func("/World/GroundPlane", ground_cfg)


        self.wall = RigidObject(cfg=self.cfg.wall_cfg)
        self.scene.rigid_objects["Wall"] = self.wall


        self.wall_mesh = sim_utils.UsdFileCfg(
            usd_path=WALL_MESH_USD,
            scale=(1000.0, 1000.0, 1000.0),
        )
        self.wall_mesh.func("/World/WallMesh", self.wall_mesh)


        self.robot = Articulation(self.cfg.robot_cfg)
        # clone and replicate