I think figured it out now, it was all an unstable overclock. (Also, I'm not even using a custom fan profile.) And yeah, the driver did crash a few times and got repaired, so when my machine freezed it was probably cause windows was unable to fix the driver crashing. So thank you, you fixed my problems with the most obvious solution (that I was unable to find myself...

<-me).
I think it's alarming though that you can clock your card too high and still see no graphical bugs until suddenly it just crashes. Or maybe I'm just getting confused because I'm usually not much of an overclocker.
Anyways, thank You for Your help!