cleaning up $PATH in bash

My path has a lot of entries that were added long ago by scripts. They are not in my .bashrc, .bash_profile, or .bash_login.

I'm worried that resetting my path in .bashrc will have undesirable long-term results. Is there a way to find where things have been added to my path and remove them manually? Are things always added by file or is path cached somewhere? If the latter, is it easy to clean that up?