Deselected stuff are still selected

When I have several things selected in the Outliner and I Ctrl+LMB one of those things, it becomes the active object, if I hit Ctrl+LMB again it gets unhighlighted but it is still selected, because when I enter UV editor, it's UVs are still there, meaning it can't be deselect through this method.

That happens because the first time you Ctrl+LMB the object it becomes the active object, my suggestion would be to deselect it with the first Ctrl+LMB click, and if my intension is to make the object the active one, then I Ctrl+LMB a second time, at least in the Outliner. That would be more intuitive then making it the active one first and spoiling the deselection second.