The number of cuda cores does matter. That's like saying a six core cpu is as good as a quad core at multithreaded apps. The performance gap is bigger than the 980 vs 970. You have 2 gigs more, a larger bandwidth bus, and 800 more cuda cores. 970 is missing about 500 cuda cores and ~0.5gb ram but it's on the same architecture. So it's not like there's that much difference. 980 vs 960 is about a fair comparison as 980 vs 980ti. Wow now I really feel screwed.
The problem is that the extra six gigs of memory in the Titan x won't be of much use for at least 6 months or so. We haven't even seen games running dx12. My guess is Titan x was a preemptive step for dx12. I also think that nvidia knows what the r9 300 series is looking like on paper and they priced the 980 Ti about $100 lower than it was supposed to be so they could grab as much of the market share as they could before the 300 series came out.