Dabadger84
Yeah you would hear it.  That's the same sensor glitch that's occurring with most 30-series cards that have iCX sensors I'd wager - randomly spikes that only last a split second (and aren't really happening) to 6500+C on the readouts, and fans going to crazy numbers for a split-second as well.  It's only a sensor/readout glitch, the fan isn't actually spiking that hard (pretty sure it's actually impossible for them to spike that hard).
 
I would highly recommend you setup a custom fan curve with steps instead of letting the card control the fans itself.  Set the highest speed it will go to to a speed that is just under "Too loud" for you, and set that speed to take place at around 60C, then step it down 25% or so for less than 60C, and so on.  This is how mine is setup, but it should be noted that A: I have great airflow, B: It's a Kingpin card on the 360mm AIO cooling & C: my temps never go high enough to trigger the 100% fan speed:
 

 
You can do the same sort of steps in Precision X1, I believe you just have to double click the fan control graph & it will change from how it is be default to "steps".  The reason behind doing the step-method instead of a gradient is it results in less fan-speed changes - the part that's most audible to most people is constant changes in speed, the less often your fan changes speed, the less you'll notice.

Dabadger84
This particular issue, if it is the sensor spikes, is an NVidia Driver issue, unsure if eVGA can do anything to fix it since they didn't cause it in the first place.  It was not present on the initial release drivers, but then was on every driver since then, from what I understand.
 
Apparently Kingpins are immune to it, because I haven't had the issue once since getting mine & running it on the 461.40 drivers.