What you described is not only eerie, it's very similar to a situation I had (and am still experiencing). About a month ago, I found out that my friend was coming here often to another guy's apartment without me knowing. Over the course of 2 nights, I was so just outraged at hearing them both talk (thin walls), suddenly, I was able to tell her things, and vice versa.
Now, I KNOW THIS SOUNDS INSANE, but there are two people in my apartment building that wake me up, remind me I have some serious issues, but I also "look" for their voices in the morning.
I KNOW that they can hear me, and to be frank, I'm really scared of what this will turn into if it doesn't stop. If you found any answers, please email me at devontowers@yahoo.com
Thanks for your courage and post (holy shit),
Dev