Your supply will naturally increase with your baby`s. If he`s already 4 weeks old and of good weight and getting enough wet diapers, I don`t think you have to worry so much.
The late afternoon is a natural time to be a bit `low` in milk, as you are tired and the baby`s been feeding all day. Apparently though `low` this milk often contains the hindmilk. But your baby might enjoy to cluster feed at this time anyway, so he can go a bit longer in between feeds in the evening.
When my son was first born and I was also worried about my milk supply, I just pumped a lot. But I ended up building up too much of a supply and I had some oversupply issues that lasted a while and made difficulties with feeding my baby.
I think natural is best, what you are already doing, feeding on demand.