Shek O is not really very far from Repulse Bay in ocean terms ie it is the same body of water and probably a similar distance from the shipping lanes. There can also be a huge amount of rubbish in the water in both Big Wave Bay and Shek O. As mentioned above a lot is to do with the winds and the tide. My observation is that this time of year when winds are from the north and the air pollution is particularly bad, the water quality appears better ie. less rubbish and clearer. But that is observation only.
Our kids (and us) swim in the water at both Shek O and Repulse Bay - well normally South Bay but same same and have swum at Repulse Bay. They are a bit older though but youngest was in there around 12 months. If you believe the governmnet rating of the water of the water it is more consistently higher at Repulse Bay compared to Shek O.