I think it depends on the child as far as how much sleep they need (and if he's ready for the day). My daughter will be three in October, she still goes to bed at 7pm, sleeps till 7am and has a two hour nap in the afternoon. The best gage for me as to if my kids are getting enough sleep is whether or not they are happy and well behaved.
You might try BCL's idea of doing a quick feed and change and seeing if he'll go back down. If he's just playing or even fussing just a bit, I'd turn leave him to play in his crib. Otherwise, maybe try shifting his whole schedual a bit later till you've got a better balance. I know its difficult to push bedtime when they seem so tired, but it doesn't have to be all at once. Try pushing bedtime 10 or 15 minutes later for a few days in a row. once they've adjusted to that, try pushing it again.
A 20 minute morning nap seens pretty short. He's probably wiped from getting up so early that he's taking a little cat nap. He might be ready to drop that nap if it weren't for the early mornings.
I'd agree with DTHY and particularly try to shift that afternoon nap till a little later. One trick that I once read about afternoon naps is to get your child in the habit of going down (starting your nap ritual) right after lunch. The'll start associating the two events together. It's then easy to have lunch a little earlier or later if you want to modify their nap schedual and they don't notice the change in routine.
Every time you get into a good groove something changes (teeth, sick, dropping naps, jet lag). Just remenber that you will get through this phase as exausting as it may be.