The Coastal Defence Museum on the Eastern Side is a place where they can run around in the moat and walk through an old fort. The Fireboat in Quarry Bay is also ok for kids to play on.
Heck, with my kids, just riding the tram back and forth across the city is sensory overload for them. Counting cars, buses, lights, cameras, etc. They have fun sitting in the top seat just looking out the window.