Twitter just added a new feature that lets you add a seemingly infinite number of people to a conversation without affecting the 140 character limit. You can, essentially, @ everyone at once. It's fine when it's just a few people tweeting back and forth, but more annoying tweeters can take advantage of the feature and blow up your notifications.
In short, you just need to mute the conversation. Whether you're using the mobile app or the desktop site, just open one of the threaded tweets and click the upper right corner:
Then you can elect to mute the whole reply-all catastrophe (otherwise known as a "canoe" in insidery Twitter parlance). You can do the same thing in the mobile app. And then the dozens of other people in the thread that are constantly replying just for the sake of filling your notifications won't pop up any more. (Technically it looks like the number of people in a conversation is limited to 50.)
What I learned firsthand this morning from being included in such a thread is that 1) I had to turn off notifications on my phone, as the constant stream would probably kill the battery and 2) I otherwise didn't really care because I just ignored my annoying colleagues. But if Twitter keeps this feature as is, it could easily become another tool for the trolls who still plague the site. You can also set up some more advanced filters to keep dummies out of your mentions.