Planar Ballbot, Balance & Station Keeping

A tall robot body balancing on a driven ball, one vertical plane of the CMU ballbot architecture, trained to hold a floor position, transit between stations, and recover from shoves. A discrete LQR and a PPO policy are benchmarked head to head on identical station-keeping trials.

OVERVIEW & MOTIVATION

Ballbots trade the single line-contact of a Segway for a ball, giving omnidirectional motion but adding the ball’s own rotational inertia to the balance problem. This project isolates one vertical plane of that dynamics (the full 3D system decouples into two orthogonal planar systems plus yaw, which is also how the real CMU controllers are designed) and asks a station-keeping question on top of the usual balance question: not just stay upright, but hold, and move between, floor positions under torque limits, deadband, and quantized sensing. Part of a five-plant series on RL for underactuated control.

PHYSICAL SYSTEM & PARAMETERS

State is [x, v, theta, theta_dot] (ball contact position, its velocity, body lean, lean rate); the single action is roller torque on the ball through a no-slip drive. Dynamics are the coupled mass-matrix equations for a ball-body pair, integrated with RK4 at 100 Hz; energy conservation was verified to 3e-8 with dissipation disabled. The ball moment of inertia uses the hollow-sphere formula I = (2/3) m r^2; body inertia is a rod about its centre of mass.

PARAMETER	VALUE	UNIT
BALL MASS (m_ball)	0.60	kg
BALL RADIUS (r)	0.11	m
BODY MASS (m_B)	2.5	kg
BALL CENTRE TO BODY CoM (l)	0.30	m
GRAVITY (g)	9.81	m/s2
DRIVE FRICTION (b_x)	1.0	N s/m
BODY PIVOT FRICTION (b_th)	5e-4	N m s/rad
MAX ROLLER TORQUE (TAU_MAX)	1.2	N m
SIMULATION RATE	100 (RK4, 5 substeps)	Hz
ODOMETRY QUANTIZATION (X_QUANT)	0.001	m/count
IMU LEAN QUANTIZATION (TH_QUANT)	0.1	deg
EPISODE LENGTH	2000 steps / 20	s
FALL THRESHOLD	0.5 (~29)	rad (deg)

METHOD

Observation is the quantized state plus the station error, [x, v, theta, theta_dot, x - x_ref], five dimensions. Action is a scalar in [-1, 1] scaled to plus/minus TAU_MAX roller torque, zeroed inside a deadband of 0.05. Station targets step randomly every 400-800 steps (4-8 s) to a new position in [-1.5, 1.5] m, and the observation always carries the current position error so the policy can plan the transit.

The LQR baseline linearizes the mass-matrix dynamics numerically about the upright equilibrium and solves the discrete algebraic Riccati equation for the state-feedback gains at 100 Hz.

Reward is cos(theta) - 0.4*(x - x_ref)^2 - 0.02*v^2 - 0.02*a^2 - 0.01*theta_dot^2, with a -5.0 penalty on falling (|theta| > 0.5 rad). This weighting replaced an earlier version (position weight 0.15, velocity penalty 0.05) under which the learned policy discovered that standing still was cheaper than travelling and parked 0.5 m short of far stations, a quiet RL failure mode fixed by re-weighting position error up and velocity penalty down. PPO (Stable-Baselines3, MlpPolicy) trained for 2,000,000 timesteps with n_steps=512, batch_size=1024, learning_rate=3e-4, gamma=0.99, gae_lambda=0.95, ent_coef=0.005.

RESULTS

Evaluated over 20 episodes each, 20 s, random station targets:

METRIC	LQR (DISCRETE)	PPO (2M STEPS)
SURVIVAL	20/20	20/20
MEAN ABS. STATION ERROR (INCL. TRANSIT)	0.235 m	0.338 m
SHOVE RECOVERY (theta_dot += 1.0 rad/s)	20/20	20/20
LIVE DEMO: SETTLE AT A 1.2 m STATION	3.0 s	2.3 s

The LQR holds station tighter; PPO settles a 1.2 m move faster in the live demo and, after the reward re-weighting, settles inside plus/minus 0.10 m. Headless verification of the live demo further logged a 1.2 m station settled in 2.29 s and shove recovery in 0.92 s for PPO.

USE CASES & APPLICATIONS

Ballbot-style balancing is the mobility base behind omnidirectional service and hospital robots, where a small footprint and dynamic stability matter more than static support-polygon size. The same station-keeping problem, hold position, accept a disturbance, retarget on command, maps directly onto dynamically stable mobile manipulation platforms that need to reposition precisely between tasks rather than just avoid falling over.

FILES & REPRODUCTION

ballbot_env.py    custom Gymnasium env (all physics here)
lqr_baseline.py   numeric linearization -> discrete ARE -> gains + eval
train_ppo.py      PPO training (Stable-Baselines3)
evaluate.py       LQR-vs-PPO table, shove test, GIF
live_demo.py      real-time interactive demo with station slider + results sheet

pip install -r requirements.txt
python lqr_baseline.py    # discrete LQR: 20/20, prints gains
python train_ppo.py       # ~15 min CPU, saves ppo_ballbot.zip
python evaluate.py        # head-to-head table + results/ballbot_ppo.gif
python live_demo.py       # interactive demo (--headless N to verify)