We loved San Diego. Great spot with kids. We spent two weeks in La Jolla and took it easy, going into San Diego to see some of the things you mentioned.
In my opinion each of the outings needs a day with a 3 year old. Not necessarily a whole day but by the time you get there, see what you need to see, eat etc you'd have to rush to do anything else.
If you can't spend that amount of time there maybe scale back a little otherwise it all might become stressful with you rushing everywhere and worse, a blur for your child!
Have fun.
