what goes around comes around

The worst thing about having one night stands or meeting some random people for sex is when they call later after some days and you don't feel like meeting them anymore. I'm a conquerer so as soon as I get someone for sex I'm no longer interested in this person for sexual reasons later (with some exceptions). And I hate when they call! It's ok for me to call a guy for the second, third,... time because it's my choice etc But why the fuck is he calling me? Did I ever ask for that? It's often so annoying. And I need to work on my assertiveness in order to say that "I'm just not into you anymore, baby"... Or maybe it's better to lie? Any suggestions?