I’m currently building up a compute rig for LLM experiments using a Supermicro X9DRG-QF, two E5-2697v2, 1TB of memory and a rack of old Tesla M40 24GB cards. Two EVGA 1600 P2 for power and two EK AIO 280 for CPU cooling. A decade old example of parts machine CUDA compute power at work. Got some serious deals on the hardware, so it’s been a cheap build so far.

Currently set up for Ollama and Open-WebUI use. Running Llama 3.3 70B FP16, QwQ 32B FP16 and Llama Vision 3.2 90B Q8 models.