Autonomous Stack
with IPG CarMaker
FS-AI 2026 Dynamic Driving Task -ADS-DV Platform
What We'll Cover
What is IPG Formula CarMaker?
- Full-vehicle simulation platform by IPG Automotive
- Physics-based tire, suspension & powertrain models
- Sensor simulation: camera, LiDAR, radar, object sensors
- MovieNX 3D rendering with road surface models
- Formula Student vehicle library (ADS-DV ready)
Key Features
Our Setup
Why CarMaker for FS-AI?
| Aspect | Real ADS-DV | CarMaker Simulation |
|---|---|---|
| Cost per test | $$$ -track rental, team, logistics | Free -unlimited runs |
| Sensor data | ZED2i stereo + IMU + wheel encoders | Synthetic camera + object sensor + ground truth |
| Crash risk | Real damage to vehicle | Reset in seconds |
| Track variety | Limited to available venues | Any track layout via Tcl scripts |
| Repeatability | Weather, tire temp, battery SOC | Deterministic conditions |
| ROS2 integration | Native (same stack runs on car) | Same ROS2 stack via CMRosIF |
Same ROS2 nodes run in both environments -simulation is not a separate codebase
Preferred over Gazebo, LGSVL, CARLA -CarMaker provides validated vehicle dynamics, official FS-AI vehicle models, and deterministic sensor simulation out of the box
What is CMRosIF?
The CarMaker ROS Interface -a bridge that embeds ROS2 publishers and subscribers inside the CarMaker process itself.
Physics Engine
libCMNode_ROS2.so
Topics / Services
How it works
- CMNode is a shared library loaded by CarMaker at runtime
- CMJob scheduler syncs ROS publish/subscribe with sim clock
- No standalone process -lives inside CarMaker's main loop
What it publishes
How CMRosIF Attaches Our Bridge Node
How the Full Pipeline Starts
perception → YOLO cone detector
slam → EKF-SLAM + wheel odometry + TF
planning → ft-fsd path planner
control → Pure Pursuit controller
rviz2 → visualization
Config: Data/Config/CMRosIFParameters -launch args: perception, slam, planning, control, rviz2
YOLO Cone Detection
ZED2i stereo camera gives us RGB + depth. YOLO12 finds cones in the image, then we look up each cone's depth to get its 3D position in the world.
- Input: 1920×1080 RGB + aligned depth map
- Model: Custom YOLO12 trained on FSOCO dataset
- Confidence threshold: 0.55, max range: 25 m
5 Detection Classes
sample_depth() from our code
if bw > 15:
sx1 = x1 + int(bw * 0.3)
sx2 = x2 - int(bw * 0.3)
roi = depth_image[y1:y2+1, sx1:sx2+1]
depths = roi[roi > 0].flatten() / 100.0
# IQR outlier removal
q1, q3 = np.percentile(depths, [25, 75])
iqr = q3 - q1
lo, hi = q1 - 1.5*iqr, q3 + 1.5*iqr
inliers = depths[(depths >= lo)
& (depths <= hi)]
return float(np.percentile(inliers, 10))
Perception Data Flow
What is EKF-SLAM?
Breaking Down the Name
SLAM = Simultaneous Localization And Mapping. The car figures out "where am I?" and "where are the cones?" at the same time.
EKF = Extended Kalman Filter. A math recipe that combines noisy guesses into one better answer.
The Analogy
Imagine walking blindfolded, counting your steps (wheel odometry). You drift over time. Now you can occasionally remove the blindfold and see the landmarks around you. A Kalman Filter blends your step-counting with those landmark sightings, correcting your drift every time you see something familiar.
How the Kalman Filter Works Inside
Every value we track has an uncertainty (a number saying "how sure are we?"). In the predict step, we use physics to guess the next state, but uncertainty grows. In the update step, we compute a Kalman gain K between 0 and 1. K = 0 means "trust prediction", K = 1 means "trust the sensor". The filter automatically picks K based on which source has less noise right now.
State Vector (what the filter tracks)
vehicle = [x, y, θ] // 3 values
// Every cone's position on the map
cones = [c1x, c1y, c2x, c2y, ...]
// 2 values per cone (x, y)
State = [vehicle | cones]
// Total: 3 + 2N values
// 50 cones = 103 numbers tracked
Covariance Matrix P
Alongside the state, we keep a covariance matrix P (size (3+2N) x (3+2N)) that stores the uncertainty of every value AND how they relate to each other. When we correct the car's position, the cone positions get corrected too because P links them together.
How SLAM Builds the Map
The 5-Step Loop (every frame)
Data Association (from our code)
for detection in observations:
costs = []
for lm in landmarks:
dist = norm(lm.pos - detection)
if dist > 5.0: # too far
costs.append(INF)
else:
costs.append(mahalanobis)
# Hungarian algorithm solves assignment
Subscribes
/perception/detections_3d/localization/wheel_odom
Publishes
/slam/odom/slam/global_map
/slam/tracked_cones
Params
range: 4-25 mmin_hits: 3
σxy: 0.3 m
Path Planning Fundamentals
Given sorted blue (left) and yellow (right) cones, find the driveable centerline.
FT-FSD Path Planner
We use ft-fsd-path-planning (MIT license). No Delaunay triangulation - it uses trace-based sorting and distance-angle cone matching.
3-Stage Pipeline (Trackdrive)
Skidpad Special Algorithm
Skidpad is a known figure-8 layout. Instead of computing a new path, the library:
Acceleration mission uses a similar relocalizer for straight-line paths.
Our Custom Path Planner (Design Goals)
ft-fsd works but has limitations. We want to build our own planner with these goals and considerations:
Design Goals
- Work with as few cones as possible (even 1 per side)
- Graceful degradation: partial path is better than no path
- Use rear axle position (Fr1A + 0.5637m) as reference
- Multi-lap path stitching: connect end of lap back to start
- Curvature output for speed profiling by the controller
- Replanning at 10+ Hz without jitter or oscillation
Edge Cases to Handle
- Only cones on one side visible (tight corners)
- False positive cones from perception (outliers in map)
- Orange cones at start/finish line (not part of boundary)
- Sharp hairpin turns (>90 deg) where sorting breaks
- Lap closure: detecting we've completed a full loop
- Path continuity across replanning cycles (no sudden jumps)
Open Questions
Should we use curvature-aware splines? Can we encode track width as a safety margin? How do we handle dynamic replanning without path oscillation?
Pure Pursuit Controller
Think of a dog chasing a ball -it always runs toward the ball. Pure Pursuit works the same way: pick a "lookahead" point on the path ahead, then steer toward it. The faster you go, the further ahead you look.
Steering Law
ld = 0.8 + 0.3 * speed # [0.8, 3.0] m
# Transform target to vehicle frame
local_x = dx*cos(yaw) + dy*sin(yaw)
local_y = -dx*sin(yaw) + dy*cos(yaw)
# Pure pursuit formula
steer = atan2(
2 * L * local_y,
local_x² + local_y²)
# L = 1.53m (wheelbase)
Speed Profiler (forward-backward)
v = sqrt(a_lat_max / |κ|)
# 2. Forward pass (accel limit)
v[i] = min(v[i],
sqrt(v[i-1]² + 2*a_max*ds))
# 3. Backward pass (brake limit)
v[i] = min(v[i],
sqrt(v[i+1]² + 2*a_brake*ds))
±0.3665 rad (21°)
kp=1.0 ki=0.15 kd=0.05
Control Mode Switching
Two modes: Autonomous (Pure Pursuit follows the planned path) and Manual (human drives via Tkinter GUI). Switch between them via a ROS2 service call at runtime.
VehicleControl Message
float64 gas
float64 brake
# steer_ang = road wheel angle (1:1)
# gas/brake = [0.0, 1.0] normalized
Pedal Mapping (from code)
torque = f_need * wheel_radius
gas = torque / (120 * 3.5)
Future: Safe-Set Trajectory Optimization
After completing lap 1 with Pure Pursuit, we have a proven safe trajectory. Now we can use it as a safety net to push faster on subsequent laps.
The Concept
Top-Down Track View
Full Pipeline at a Glance
YOLO + Depth Wheel Odom
IMU + Encoders
What's Next & Open Problems
👁 Perception
- Real ZED2i stereo depth vs. simulated
- Cone classification accuracy in rain/glare
- Latency optimization on AGX Orin
- Training data augmentation pipeline
🗺 Planning
- Sharp corner handling (>90° turns)
- Lap closure detection reliability
- Multi-lap path optimization
- Dynamic re-planning at higher speeds
🎯 Control
- Safe-set trajectory optimization
- Tire model integration (dynamic bicycle)
- Sim-to-real transfer gap
- Emergency braking logic
Thank You
Ozyegin University • Istanbul, Turkey • FS-AI 2026 DDT
References: IPG CarMaker User Guide • ft-fsd-path-planning (MIT) • FS-AI Rules 2026