You can get record players at your local music shops or online if you prefer.
Since you'd just be using it to play records, not DJ, then you can get a standard record player without having to worry about it meeting some kind of performance demand. That in mind, you could probably find what you're looking for between 50-100 us dollars. You might also get good deals on used players.
Yes, there's plenty of places that sell vintage and new records. You've probably got shops around town, and there are definitely online shops.
If you would get enjoyment out of collecting vinyl and listening to them, then yes, it's worth it.
__________________
|