Sing to God (1995) by the Cardiacs
It's a fantastic double album that combines elements of progressive rock and punk rock, but... it's actually done
well. I have to say that I fell in love with it upon the first listen, and I am forever indebted to Anteater for recommending it to me.
I went ahead and downloaded
A Little Man and a House and the Whole World Window (1988).