Click outside the manipulator = Selection

I would like to see this feature as an option. I prefer to use the Transform Tools with handles, however, when they are turned on, I need to turn off the Transform tool in order to change the selection and then turn it on again.

To make handles visible constantly, regardless of which tool is used, it seems to me not a very good idea. It is confusing and hinders the selection.