It has often made me wonder, though, why was God angry that Balaam went with the men, when Balaam sought God's guidance and was told to go with them. Didn't he first say no? Then God told him to go and he went? Then God was mad that he went?
Here's what I take away from that encounter - once God has said, "No" in a way that is unequivocal, do not keep asking or He may say, "Yes," and you may be sorry that He did. It reminds me of a child badgering their parent for something the parent knows is not going to turn out well. They may eventually give in and allow the child to experience the consequences. Some of us have to feel the sting to learn the lesson.
There are plenty of things that God has said "no" to in the Bible that we do not need to pray and ask His direction on. We know the answer. We may not like it, but we do know.