The autopilot system in cars has to be able to map information about the environment that can be used to establish context and make judgments about pedestrian and group behavior, as well as different city intersections.