it's less that they are separate and more that they are a venn diagram. they can all be under the umbrella of "baneful magic". jinxes, hexes, and curses are varying degrees of baneful magic.

a jinx tends to be something that causes an annoyance. a hex is something that causes a consistent affect over the person who it is cast on, and typically (though not always) causes greater disturbance or disruption. a curse is something long lasting - it's not just about one person, it's about the generations to follow (and in some cases the generations that preceded) the person who is cast over.

domination magic can be a jinx, hex, or curse. domination magic can also be something behavioral - ie changing the behavior of the person you are casting over, without necessarily causing them discomfort or harm.

one isn't necessarily better than the other, it's more about choosing the appropriate spell to match the situation at hand. the degree of the spell should in theory (though doesn't have to bc what are rules anyway) match the degree of the injustice or harm caused to the person casting the spell