Make "X" when nothing is selected consistent

I'm always being told that "of course that's hidden, you're in the wrong mode". Fine.

If NOTHING is selected, can we please disable "X"? It's the wrong mode, ie nothing is available to Delete, so the dialog shouldn't appear at all.

This at least would be consistent with Blender workflows.