UPDATE: Sunday, December 8th 2024 The front page has been
updated with new information!
(excerpt / copy)
A long overdue update!12.08.24, 5:43pm CST
I was emailed recently about how long the updates were taking by a kind folder, and to be honest I've been so busy lately I haven't had the time to check on that kind of stuff so it just flew under my radar. The database size was getting up there and updates were taking well over an hour! I decided to remedy this in two stages.
First, I bumped up the new minimum user tracking amount from 550 points to 5,000 points. I think there is only one or two WUs that are below this amount, but a quick scan through my data was showing me there weren't any active users below the 5k threshold so I went with it. This took several days to run, and would automatically pause for update cycles. I ended up pruning out about 183,000 idle users and over 600 million rows of data, reducing the user history table size by 27%! This alone dramatically sped things up, but there's still more I can do!
Next, I finally worked out the correct logic to prune out rank change records only for idle users, without affecting the recent movement, or the historical records if / when they did submit WUs. Again, this is simply pruning out records where they are idle and dropping down in rank. I ran this code fine on my dev server here at home, I still need to add a few tweaks but will likely get that going within the next day or two.
I know I said over a year ago I was going to update the ranking color scale! Sometimes life just keeps you busy! I will try my best to work on that too once I get the above mentioned script running.
As of now the stats updates are back to taking less than 15 minutes to run, a vast improvement over the hour-plus from previous. I think I'm going to try to add to the database stats page like a table showing recent update processing times and / or some averages or something, since every update cycle the start / stop time is recorded.
Also, I've been recording the OS total data for years now, just the raw data sitting in a table. It's the numbers showing Windows, Linux, and Mac, along with AMD, Nvidia, and CPU totals. I will also try to get a page or two with that up at some point.
Finally, I would like to give a HUGE thank you to all the recent donations from people! Ad revenue has really dropped off a cliff since the beginning of this year (to almost nothing) so I'm having to pay for most of the hosting cost out of pocket and the donations are really really really appreciated!