Also consider the seeds and peers.
If a file you're downloading doesn't have a lot of peers seeding it, you don't have as many sources to download from.
For example:
If I tried searching for an obscure band that isn't hosted by many people and has very little seeds, I won't get the speed results compared to a download that does.
__________________
|