TiN_EE
What I can add is that, most of this memory access/routing/operation is not relevant to X299 Dark specific, but apply on total for a given platform. If you look on performance of one board with chipset X and CPU A, then you will see majority of all other boards on chipset X and CPU A doing same thing, given everything runs on same clocks/settings.

 
Logic implies that is the pattern one should see but I didn’t have data that would tell me is that what I should be seeing.
 
However just because we have, I believe, established there can be some drop in some benchmarks when going from two to four memory channels that doesn’t, I think, yet mean everything is confirmed as kosher and performing as it should be to its full potential, period (in my setup at least).
 
Reason why I say this is one of results I have seen but haven’t reached point of posting about.
 
To be specific: Spreadsheet part of PCMark10 test. It has shown 20% performance drop when going from 2 to 4 channels while all other parts of PCMark10’s tests showed practically no change (well, withing few %).
 
Observation of its execution indicates spreadsheet part is yet another single threaded “benchmark” whose result is linearly proportional to speed of “CPU” (when I increased frequency X% result would increase practically same amount).
 
Because clock speed was same in 2ch and 4ch test that, I believe, indicates only reason for getting 20% drop is something that happens when going from 2 to 4 channels.
 
As I experimented I tried hyperthreading off and figures for spreadsheet part went UP to the point where ones for 4 channel with HT off were higher than ones for 2 channel (both with HT off or on)!!! All other PCMark10 results didn’t change, neither up nor down, by turning hyperthreading on/off.
 
I repeated few other tests and naturally for some of them results dropped when turning off HT. Cinebench R20 MT was ~ 20% slower (which is actually not that much for cutting number of CPUs in half).
 
Results of y-cruncher were slightly mixed bag. Turning off HT actually improved results of 2 channel run. Not significantly but considering it had two time less “CPUs” to work with that is, I think, huge. 4 channel run was slightly slower with HT off than on but again I believe, considering it had twice less CPUs, that is still great.
 
What are your thoughts after all of this? Is it possible that hyperthreading is one of contributors to me having big drops in some tests and if yes why? To me, as an uneducated layman, first thing that comes to mind when I think “hyperthreading … (memory) performance improvement with ht off / drop with on” is “security mitigations of side channel attacks” but I am now in way over my head and would love thoughts from person that knows way better.

I am not a software or driver engineer to know lot about how threading and HT in particular works on software level, but so far your findings are in line with what known in community, HT is not always improving performance, sometimes it works the opposite. There are many applications, especially older ones, that scale only to limited amount of cores, and after that don't scale at all. And if that limit is below your actual real CPU cores, than you get into interesting situation with HT.
 
For example from 3D benchmarking - there is app with scaling to 32 cores only for example. Meaning if you have 64 cores, half of the cores would not be utilized at all. 
Now if you have 18 core CPU, and HT OFF, all cores would be used, and giving best they can for 18 threads operation. Nice predictable scores, no problems. Now you take same 18 core cpu with HT ON. Well, now each core is "interleaved" with HT logical core. You would think benchmark will use 18 real cores + extra 14 HT cores, but instead windows scheduler will be loading only 16 real cores and 16 HT cores, and 2 real cores + 2 HT cores will be unutilized. And because all of this dynamic , you get big variations in benchmark and performance could be worse than actual 16 core CPU + HT ON ;)
 
And not all loads scale same way. Rendering scales great, because it's easy to split work into independent pieces and have each CPU work it's bit. But it's memory intensive. Computation often scale good too, and may be not as memory hungry. But other apps that must use results from previous computation, more like a serial chain compute - cannot scale to many cores well, so you'd see different picture and performance footprint.

TiN_EE
There are many applications, especially older ones, that scale only to limited amount of cores, and after that don't scale at all. And if that limit is below your actual real CPU cores, than you get into interesting situation with HT.
…
And not all loads scale same way.

 
That is true so PCMark10 spreadsheet part on its own wasn’t something I paid high value to.
 
y-cruncher I tried is (so I hear) supposed to scale well with number of threads (Anandtech articles show high increase in performance when it multithreads, even with number of threads much higher than mine), but it didn’t scale. By then I started thinking “Why it didn’t scale when I hear it should? What might be holding it back? Well, at least it didn’t drop, but …” It is possible I have tested incorrectly so I will have to RTFM better and repeat that test and if it is still not scaling up with ht on reach out to author for comment.
 
It is Prime95 benchmark drop though (please see my previous post, reply to xuqi99) that got me completely thinking. Should it be scaling up instead of dropping? I don’t know. Is it “normal” it drops? I don’t know, maybe not, maybe yes. But even if yes is 27% drop induced by ht normal amount? I couldn’t find answer on that so I will have to reach out to P95 community.
 
In the meantime Anandtech articles indicate DigiCortex benchmark is both intensive on memory and scales real well with hyperthreading on so I will have to look into setting it up. And if DigiCortex too shows no scaling or drop then we have a definite “what here is off” …

I’ve run complete SPECworkstation suite (except for, of course, storage benchmark). Results are:
 
Majority of tests showed no practical difference in performance between 2 channels and 4 channels with hyperthreading on. When they did drop going from 2 to 4 channels drop was usually in 1%, small number was in 2% range, and only one (srmp) was 4%. We can say that the average of the drops bigger than measurement error of 2% was 4% with single data point.
 
On the other hand number of benchmarks that showed performance increase when going from 2 to 4 channels was much bigger. 19 of them had gain bigger than 2% and their average was 29.4%.
 
I’ve also re-run SPEC suite with hyper-threading off (4 channels). Good number of results showed neither scaling up nor down and I didn’t have time to check are those the ones that are single-threaded or not. Several of them dropped more than 2% with worst one (Poisson) dropping 31% and second worst (fsi Binomial) dropping 12% even though both are multi-threaded. Remaining “drops” had an average drop of 4.5%.
 
On the other hand number of benchmarks that showed performance increase with hyperthreading was much higher. Some scaled up real good, some not so good, and average gain of those that had gain bigger than 2% was 25.9%.
 
{… to be continued …}

Had I read past the first page before I posted the above, I may not have posted it.
Okay, now I have read it all and apologies if I missed it but there is one test that you may have not run - real world tests on real, large datasets.  Sorry, I forget exactly what you said you do, but in doing what you do, I imagine you could come up with multiple  examples of repeatable data runs.  Yes?  Big tests with enough data to remove the cache from the equation.  If you did that with as much rigor as you have applied to canned benchmarks, I suspect this phenomenon will fade from your concern.  I hope so.

I’ve done further looking at DigiCortex and y-cruncher but haven’t had time until now to write it up. For those that are interested conclusion was:
 
a) I don’t feel DigiCortex should be used as a benchmark for anything. At least I won’t.
 
b) y-Cruncher does show memory bandwidth going up from 2 to 4 channels -but- I don’t feel it can be used as a benchmark tool for memory or anything else. Developer himself says “Long memory latencies are hidden away fairly well by hyperthreading” and “On Skylake X processors, L3 cache bandwidth is also a bottleneck. So overclock the cache as much as possible.” Only other piece information I was able to get by spending effort on y-Cruncher was that my i9-10900X had same performance in it as i9-7900X running at same speed.
 
That leaves checking why Prime95 bandwidth benchmark returned 30-ish% better figures with hyperthreading turned off.
 
{… to be continued …}