For many particular examples, there have already been enough rathole spelunkers to provide useful data. Maybe start looking in the places where there isn't already useful data?
Agreed. It's often much, much harder to articulate why an idea is bad or a rat hole. You just move on.
I've come up with explanation by analogy. You can demonstrate quite easily in mathematics how you can create a system of notation or a function that quickly becomes impossible to compute. A number that is too large, or an algorithm that would take infinity amount of time and resources to solve...
It seems to be in nature that bad ideas are easy. Good ideas are harder, because they tend to be refinements of what already exists and what is already good.
So pursue good ideas. Pursue the thing that you have thought about and decided has the best balance between values and highest chance to succeed. Sometimes it's just a strong gut feeling. Go for it, but set limits, because you don't want to fall prey to a gut feeling originating from strong intuition but an equally strong lack of fundamental understanding.