I tend to judge my "keep holding onto or let 'em go" based on how often I see them showing up in defense teams, or if there is a costume on the way that might make them relevant again. At the point, I usually keep the newest and purge the old. Some of those were good in the day, but most aren't going to be worth the mats to take even to 4/80 let alone LB or emblem.
Another thing MAYBE to consider for who to keep is if anyone would be a good addition to your Titan team (e.g. DDown or EDD).
For what it's worth, I purged several non-dupes for Silverpaw, so I'm definitely biased that he's worth dropping deadweight heros to pick up. A further question would be if there is one more hero you'd part with to hit 20 for SE; there are some REALLY GOOD options at that level.