Wheeled recreational devices are not to be used on a road with a dividing line or median strip or on a road with a speed limit greater than 50km per hour (previously the speed limit was 60km per hour).
A person must not travel in or on a wheeled recreational device or wheeled toy that is being towed by a vehicle. A person travelling in or on a wheeled recreational device or toy must also not hold on to a vehicle while it is moving or travel within 2 metres of the rear of a moving vehicle.
(Skateboards are classified as "wheeled recreational devices".)
The 50 km/h limit might be the thing that makes skateboarding illegal at Mt Panorama (I'm assuming the speed limit there is 60).