Boards of Canada - In A Beautiful Place Out in the Country
My Bloody Valentine had a ton of awesome EPs, some of the best of any band in my opinion.
Sunny Sundae Smile
Lush had some fantastic early EPs; Gala was a collection of their first 3 and maybe their best release (they turned crappy later). It included Scarlet, my favorite song by them, and a lot of their noisier stuff!
Lastly, the Lazarus EP by the Boo Radleys could be my favorite EP from any band. It was the perfect combination of all their styles, and ran together perfectly.