Breaking one's bay'at

I took bayah a number of years ago to a sufi Sheikh in the Shadhili tariqa. I have recently been besieged with doubts about the tariqa. For example, I am not comfortable with the hadra which is a big part of the tariqa. I also frequently people saying bayah is haram (I know this is a minority option). Sometimes I think it is best to leave the tariqa to avoid doubtful matters. Then I worry that is is haram to break a bayah as the bayah might be considered a vow which cannot be broken. I then get very stressed because both options cause me doubt. I therefore have two questions; Is it permissible to break the bayah of a tariqa? Is it permissible to change tariqas?