Hm, now I've done some tests. I took a domain member client, started to monitor the time drift and it worked. Then I un-joined the domain and changed the credentials in the monitor to a local account, and it's still working without any hassle. So I'm suspecting that something is wrong with the target machine in the first place. Could be anything so it feels like a dead end to pursue this troubleshooting.
Rgrds,
Simon