Laziness is certainly partly to blame.
Curiosity is where the remainder of the blame lies. My knowledge of identifying trees and their scientific names is excessive; bordering on obnoxious by the opinions of friends with whom I share outdoor adventures. My skills with identifying herbaceous plants are weak but as I am actively working to improve, each of these plants offers a learning opportunity. So I generally let them go.
Side Note: my skills on identifying grasses are all but nonexistent.
In addition, most of my garden beds are fallow and contain animal manure/bedding. Root action of any kind alleviates my concerns that the beds have not composted enough to be cultivated by next spring. I certainly remove any weeds that interfere with my garden plantings. To be fair, I will probably be less generous to the weeds once my soil is ready to be cultivated fully.
Most broadleaf weeds are annuals thus not presenting much of a threat to the garden. Some of them, like false strawberry, are biennials. However the one thing they have in common is providing nectar and/or pollen to pollinators, both the native and my honeybees. Along these lines, they attract the pollinators to my crops!
This last point can be coupled with the one on curiosity- with my biological approach to farming, any one of those weeds may possess characteristics that make it a valuable tool as I found out with the unexpected Savior from Japanese Beetle Destruction.