How to Turn On UDIMs for only one object

I am modelling a character and only recently I've actually took the time to try to learn with UDIMs. My character is comprised of 3 main objects which are completelly separate: Body, Garb and Armor (And a few smaller ones)

I am trying to setup UDIMs on the body so I can give a little more closer detailing on the face and on the armor (because lots of pieces) but I don't think It is necessary on other places like the garb or the teeth, or the eyes where a single UV separate material could do. But everytime I setup a new UDIM tile It seems to apply for everything in the scene.

How can I remove/add UDIM tiles for just 1 or 2 of the objects?