Personally i'm up for getting rid of both.
Baton Pass has reached a breaking point with the advent of Speed Boost users such as Scolipede and Blaziken being allowed on the same team. Tactics such as using Roar/Whirlwind or using niche moves or items such as Red Card/Eject Button/Haze/Torment are too uncommon to be used effectively in a healthy meta game. The list of pokemon that have access to Roar and Whirlwind are very few are far between such as Hippowdon, Skarmory, or Heatran. It has gone to the point that we are even thinking about these pokemon and unique ways to stop it is making it a bit overcentralized. I'm up for a solution to reduce the power of Baton Pass or removing it entirely.
Swagger is MUCH more broken than Baton Pass. Unlike Baton Pass that actually has ways of stopping it successfully, albeit in a very unconventional manner; Swagger doesn't have much. It is mostly decided on a Coin Flip and more often than not it is towards the Swagger user's favor. Keep in mind, Swagger wasn't always so broken. Liepard was one of the kings of this tactic and was a high nuisance of this tactic. There are certain drawbacks however that set it back from being too broken with lackluster defense, outspeed by other Pranksters such as Thundurus, and weakness to Fighting-Type moves; Especially from the likes of Guts users such as Heracross, Conkeldurr, Machamp. Even Magic Bounce had a way of remeding Liepard by using HP Fighting. Now sadly, Swagger has been given to Klefki and he takes full advantage of this tactic. Better typing, better stats, better moves, and overall just a better pokemon in general is giving Swagger a bad name. Klefki has access to T-Wave, Toxic, Spikes, Dual Screens, Swagger, Foul Play, and Play Rough. Nearly all of these moves are extremely screwey and can be used in a mixed set from setting up Dual Screens to the horrible tactic known as #SwagPlay. Swagger is ultimately to blame for the problems caused by it because Confuse Ray, unlike Swagger; doesn't improve the power of confuse hax or Foul Play. Throw in the fact that Fighting Types/Magic Bounce are near useless against Klefki, it makes the tactic more deadly.