I have a GTX 980 Classified installed in a 2009 mac pro tower with an external power supply. Every so often, when using an animation program, everything locks up ad the screen goes black. Rebooting doesn't help, but force-quitting and letting the machine sit for a couple hours does. It will then reboot.  
 
Any ideas on what is causing this and how to fix it?

The program locks up...the cursor vanishes...then the screen goes black. The computer continues to hum, but there's nothing on the monitor. So I have to hold the power button down and force a shut down. If I immediately restart, I still get a black screen. But if I wait an hour or so, the desktop will appear and the program re-open.  This made me think it might be heat related, but I'm not sure.

I'm not sure how to check to temps, but I don't see or hear a huge fan-power surge when this happens. Fact is the fan rarely spins up on this GPU. The lock up seems to be triggered by rapid actions in the animation software I'm using, as though the system can't process the geometry. That's another suspicion. I'm not sure why this wouldn't just lock up the program and not cut out any visual to the monitor.

You can check your temps with the following program... https://bjango.com/mac/istatmenus/