I think I might have found a bug. I'm trying to cast a ray down through the world's navmesh to detect if a point is over the navmesh or not. Useful for determining if an agent is positioned inside or outside the walkable navmesh for example.
Anyway PathWorld.ray() appears to return completely unpredictable results, often producing a lot of false positives. I'm not sure if this is a bug or if I'm just using the function incorrectly. How does this function work and how should it be used?
I created a sample to show the problem. Open the Pathfind tutorial in the Game Basics folder. Then add the following code to the Draw() function... shouldn't matter where.
Code:
Vec pos, dir; ScreenToPosDir(Ms.pos(), pos, dir);
PhysHit phys_hit;
if(Physics.ray(pos, dir*D.viewRange(), &phys_hit))
{
Flt hit_frac;
Vec hit_pos, hit_normal;
Vec start = Vec(phys_hit.plane.pos.x, 5.0f, phys_hit.plane.pos.z);
Vec end = Vec(phys_hit.plane.pos.x, -5.0f, phys_hit.plane.pos.z);
if(Game::World.path().ray(start, end, &hit_frac, &hit_pos, &hit_normal))
D.line(BLUE, start, end);
else
D.line(RED, start, end);
}
Now if you run the tutorial, you will see a vertical line protruding from the cursor position on the terrain. This line visualises the ray cast test. If the line turns blue, then the ray() function has detected a hit with the navmesh, otherwise the line is red. Holding down the space will show you the navmesh that is being ray cast against... just in case you forget.
If you try moving your mouse outside of the navmesh, the line will turn blue, which means the ray cast has detected a hit... this doesn't make sense... it is outside the navmesh, how can the ray hit it?
If you try moving the mouse around inside the navmesh, the line will change between red and blue randomly for no reason. This also makes no sense. I have a feeling this ray() function might be broken.
