The same thing constantly happens to me. Sadly, It seems to have happened with Mahler's Eighth in my case. I still like it, it just doesn't make me hold my breath anymore.
And I wouldn't worry about the sixth too much. You're just one of many Nielsen fans - including myself, to a degree - who listen to it less than his others. It's no big deal. Besides, it's just one work - I still consider myself a Mahler fan. In any event, at least the third still gets you. I love how driven, yet beautiful, it can be.