The Problem With Time Travel

The problem with Time Travel is all these annoying paradoxes that crop up such as the the Grandfather Paradox where if i went back and killed my grandfather i wouldn't be born to go back and kill my grandfather so he would still be around to give birth to one half of my parents who would give birth to me who would then go back and kill him.
Headache stuff i agree but cynics say that if time travel was possible then why are we not  meeting our fellow humans from the future?
My usual answer is along the lines of how do you know we aren't, it isn't as if they will be wearing a t-shirt advertising they were part of a school-outing on a year 2013 day trip.
I am beginning to develop a new theory though around city regeneration.
Here's the science bit. The building where i live was built in the 1970's, before that it was a row of shops and before that a factory making brooms. If i developed a time machine in my kitchen and went back 50 years, i would turn up amidst tins of spam or 100 foot in the air because time moves but the machine will stay in exactly the same place.
If a professor makes a time machine in his laboratory in Milton Keynes, if he goes back any further than 1967 he will find himself sinking in a field because Milton Keynes wasn't there. If he decides to go forward in time he may well find himself sat on the outside lane of a motorway that's been built since.
There cannot be many places on the planet that a time machine can be built and can safely shift backwards and forwards while staying on the same level of ground (might end up mid air or in a hill) or guaranteed a safe landing (might end up in the sea or a train track) so the places where the school kids from 2114 can end up are very limited.
Of course if you manage to land on your Grandfather then that's a whole other problem.