up until about 4 months old, i used a hotsling with my son. it's like a fabric pouch where you can slot them in and they lie cradled against your body. when they are a bit bigger, you can sit them up in it although i didn't think this position was very comfortable for them.
you can get similar pouch slings from toys r us. about $300 i think. they fold up really small and you can just pop them in your bag when you're not using it.