Definitely worth it. There's something to be said for more cores, faster memory, and if you get an FTW or Classy, stupidly high boost speeds to boot. I would bypass getting a reference cooled card, and look for a SC or FTW (or if you can find one a classy) simply because the ref 1080 design is a bit under-powered for the 1080's heat output... it can throttle, and an FE 1080 is a pricey thing compared to even an FTW 1080.
Get a 1080, and FTW is the sweet spot for price - performance.