On learning to say no

As someone who dislikes / avoids conflict, I find myself often saying yes (when I want to say no) or not saying anything (and then regretting it). 

Sometimes in an effort to make others happy, we make ourselves unhappy. Perhaps we are trying to be agreeable but I do wonder: is it worth it?