RZ/V2N AI Autonomous Car Demo

An embedded AI reference design for autonomous navigation — powered by the Renesas RZ/V2N MPU and its integrated DRP-AI accelerator. The car detects a bowling pin target in real time and drives toward

📋 Overview

The AI Dome – RZ/V2N DRP-AI Autonomous Car is a hands-on reference design that demonstrates how real-time computer vision and on-device AI inference can drive an autonomous vehicle without any cloud dependency. The demo combines object detection, visual tracking, and closed-loop motor control to search for a bowling pin, lock onto it, and drive directly to the target.

The platform is built around SolidRun's HummingBoard-IIoT (HB-IIoT) carrier paired with the RZ/V2N SOM, extended with a four-wheel robot chassis, LiDAR, camera, and an Arduino-based motor controller.

This project is intended for developers, robotics engineers, and embedded AI practitioners who want a practical starting point for building edge-AI robotics applications on the RZ/V series platform.

---

🎯 Demo Objectives

The demo is designed to showcase several capabilities at once:

• Real-time object detection of a bowling pin using a neural network accelerated by DRP-AI.

• Autonomous search behavior — the car rotates and scans its environment until the target is found.

• Visual servoing — once detected, the car centers the target in its field of view and drives toward it.

• Fully edge-based inference — no network, no cloud, no external GPU.

• Low-power AI compute — DRP-AI delivers high inference throughput at a fraction of the power budget of a typical GPU.

---

🧠 Hardware Platform

The demo is built on the HummingBoard-IIoT (HB-IIoT) carrier with the RZ/V2N SOM, integrated into a four-wheel robot chassis with LiDAR, camera, and Arduino-based motor control.

Full assembly instructions, including mechanical drawings, connector pinouts, and step-by-step build photos, are maintained in the SolidRun developer here:

👉 V2N AI Demo: Autonomous Car on HB-IIoT – Setup & Assembly

---

The software pipeline runs cooperatively on the RZ/V2N, with low-level motor control delegated to the Arduino:

1. Camera Capture — frames are pulled from the C920 at 30 FPS.

2. Pre-processing — frames are resized and normalized for the AI model.

3. DRP-AI Inference — an object detection model (e.g. YOLOX-Nano) runs on the DRP-AI accelerator.

4. Post-processing — bounding boxes are decoded, filtered to the bowling-pin class, and scored.

5. LiDAR Fusion (optional) — RPLIDAR data is used for obstacle avoidance.

6. Control Logic — a state machine decides whether to search, align, drive, or stop.

7. Motor Commands — the RZ/V2N sends high-level commands over USB/serial to the Arduino, which translates them into PWM signals for the Motor Shield V5.3.

High-Level Data Flow

[C920 Camera] ──► [Pre-process] ──► [DRP-AI Inference] ──► [Post-process]
                                                                 │
                             [RPLIDAR] ──────────────────────────┤
                                                                 ▼
                                                         [State Machine]
                                                                 │
                                                                 ▼
                                              [Serial: RZ/V2N → Arduino Uno]
                                                                 │
                                                                 ▼
                                                [Motor Shield V5.3] ──► [4 × DC Motors]

---

🔁 Behavior State Machine

1. SEARCH

The car rotates slowly in place while scanning every frame for a bowling pin.

2. ALIGN

Once a pin is detected, the car computes the horizontal offset between the bounding-box center and the image center, then turns until the offset is within a small deadband.

3. DRIVE

With the target centered, the car moves forward and continuously re-checks the target's position, re-entering ALIGN if drift occurs.

4. STOP

When the bounding box grows beyond a target-size threshold (i.e. the pin is close), the car halts.

---

⚙️ Software Setup

Arduino (Motor Controller)

The Arduino handles the low-level PWM and encoder feedback. The firmware source lives in the SolidRun GitHub repository.

Prerequisites

• USB cable (Arduino ↔ host PC or RZ/V2N EVK)

• git

• python3

• PlatformIO CLI

Flashing the Arduino Code

bash

# 1. Clone the motor-controller repository
git clone https://github.com/SolidRun/rzv2n-arduino-motor-controller.git
cd rzv2n-arduino-motor-controller

# 2. Verify the project environment
pio project config
# Ensure a valid environment exists (e.g. `uno`)

# 3. Connect the Arduino UNO via USB and identify its port
pio device list
# Note the serial port (e.g. /dev/ttyACM0)

# 4. Build
pio run

# 5. Upload (flash)
pio run --target upload

RZ/V2N Application

The AI application runs on the RZ/V2N side and talks to the Arduino over USB/serial.

Prerequisites

• Yocto-based Linux BSP for RZ/V2N

• DRP-AI Translator / DRP-AI TVM toolchain

• Cross-compile toolchain (AArch64 GCC)

Build & Deploy

bash

git clone https://github.com/your-org/rzv2n-ai-dome-car.git
cd rzv2n-ai-dome-car

source /opt/poky/<version>/environment-setup-aarch64-poky-linux

make all
scp build/ai_dome_car root@<board-ip>:/home/root/

Run the Demo

On the target board:

bash

./ai_dome_car --model models/yolox_nano_pin.drpai \
              --camera /dev/video0 \
              --serial /dev/ttyACM0 \
              --confidence 0.5

Place a bowling pin in front of the car and power the chassis. The car begins in SEARCH, locks onto the pin, and drives toward it.

---

🧪 Model Details

• Architecture: YOLOX-Nano (or equivalent lightweight detector)

• Input Resolution: 416 × 416

• Classes: single-class (bowling_pin), fine-tuned from a COCO-pretrained backbone

• Quantization: INT8 via DRP-AI Translator

• Inference Time: ~15 ms per frame on DRP-AI

• Accuracy: >90% mAP on the custom bowling-pin dataset

Training notebooks and the custom dataset live in the training/ directory of the repo.

---

📐 Calibration & Tuning

All tunable parameters live in config/params.yaml:

• CONFIDENCE_THRESHOLD — minimum detection score to consider a valid target

• ALIGN_DEADBAND_PX — horizontal pixel tolerance before switching ALIGN → DRIVE

• STOP_BBOX_HEIGHT — bounding-box height that triggers STOP

• SEARCH_SPIN_SPEED — rotation speed during SEARCH

• DRIVE_SPEED — forward speed during DRIVE

• SERIAL_PORT — path to the Arduino serial device

🔍 Troubleshooting

Issue	Likely Cause	Suggested Fix
No detections	Poor lighting or threshold too high	Improve lighting; lower CONFIDENCE_THRESHOLD
Car oscillates when aligning	Deadband too tight	Increase ALIGN_DEADBAND_PX
Car drives past the pin	Stop threshold too small	Increase STOP_BBOX_HEIGHT
Low frame rate	Model not running on DRP-AI	Confirm the .drpai binary is loaded (not ONNX fallback)
Motors jitter	PWM mismatch or loose encoder cable	Re-seat encoder connector; verify PWM config
Arduino not detected	Wrong serial port	Re-run pio device list; update --serial flag
Car moves in the wrong direction	Motor IN1/IN2 swapped	Verify wiring against the Motor Placement table

---

📚 References

• SolidRun – V2N AI Demo: Autonomous Car on HB-IIoT – Setup & Assembly

• SolidRun – HummingBoard IIoT & RZ/V2N SOM Quick Start Guide

• SolidRun – RZ/V2N SOM Hardware User Manual

• SolidRun GitHub – rzv2n-arduino-motor-controller

• Renesas RZ/V2N product page and datasheet

• DRP-AI Translator and DRP-AI TVM documentation