yes it is fine to warm up milk. My dayhome provider used to do this and my children started to like it, so occasionally I do for them. I realize I am assuming you mean cow's milk. If you mean breastmilk, it is fine to warm that up too, of course If it is boiled likely the antibodies won't survive, but it is still completely fine nutritionally.