Funnily enough I don't think the gym leaders are that bad, I'm more annoyed by the difficulty spikes they represent. Basically you fight all these scrubs with bad moves and then comes the boss battle (including gym leader) and suddenly you face a team with good moves, good items and a mega evolution. Very big shift in difficulty.
Connor is probably the most unfair one, opening with a very powerfull lead connor can set the field to a blazing inferno quite easily. To get past him you require specific counters. Mind you I got past him with a mono-poison team which is basically a team of bug, grass and dry skin.
Aderyn is actually the most balanced out of them all. Not requiring specific counters and having a decent power level aderyn can present quite a challenge without being overly frustrating.
Ok my experiences with Tristan are not that bad since I can setup against before that loppuny shows up and wreck it. The fact that it can u-turn into loppuny really helps in that regard. I can understand why he is so difficult because it is the first mega pokemon in the game revealed.
Emily is a simple continuation from tristand and therefor you can prepare for it better.
Ok now comes rosetta the easiest gym leader in the game so far. The game is simply filled with counters against her: scizor, ferrothorn, magneton ,ninetales , weezing, nidoking, gengar, ... , sludge bomb, flash cannon, ... . You are also familiar with the field and can abuse it yourself with trhe fairy type you get from Boss.
So all in all I think that the gym leaders really make the difficulty fluctuate a bit too much.