Soap in peehole

I stuffed some soap into my peehole. It does not sting much at first, but it stays for days. Especially if it stays in longer. How long should I keep it in next time?