https://DevOpsCloud.io -- Cloud Monk Losang Jinpa, Ph.D., MCSE/MCT, GitOps DevOps Engineer

LLM PC

Make sure that your case is properly closed to prevent dust.

A common mistake most PC users make is opening the case of their unit, thinking that it will keep the system cooler. However, this might backfire and cause more issues in the long run. Dirt and dust can quickly go into your system with the case open. The build-up can clog the fans and increase system temperature, which can cause your system to overheat.

Make sure to keep the case securely closed. There’s a chance dust can still enter the system even if it’s slightly open.

https://www.cyberpowerpc.com/blog/how-to-keep-your-gaming-pc-cool-this-summer/

https://www.overclock.net/forums/nvidia.69

https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/

https://www.tomshardware.com/reviews/nvidia-geforce-rtx-4090-review

PNY GeForce RTX™ 4080 Super 16GB VERTO™ OC Triple Fan Graphics Card DLSS 3 (256-bit, PCIe 4.0, GDDR6X, HDMI/DisplayPort, Supports 4k, incl. adapter Anti-Sag Support Bracket, 3 slot) - https://www.amazon.com/PNY-Graphics-DisplayPort-Supports-Anti-Sag/dp/B0CS6XC69Y

Nvidia GeForce RTX 4080 Super review: Slightly faster than the 4080, but $200 cheaper

https://www.tomshardware.com/pc-components/gpus/nvidia-geforce-rtx-4080-super-review

Nvidia GeForce RTX 4070 Ti Super - https://www.tomshardware.com/pc-components/gpus/nvidia-geforce-rtx-4070-ti-super-review

MSI MEG Ai1000P PCIe 5 ATX v3.0 Power Supply - https://www.tomshardware.com/reviews/msi-meg-ai1000p-pcie-5-atx-v30-power-supply-review

GPT-4-level Local VSCode Copilot on a Desktop with only 24GB VRAM

https://dev.to/maximsaplin/llm-fine-tunig-on-rtx-4090-90-performance-at-55-power-29o3

https://github.com/turboderp/exllama

https://huggingface.co/blog/leaderboard-bigcodebench

ktransformers_vs_llama.cpp.mp4

Local 236B DeepSeek-Coder-V2: Running its Q4_K_M version using only 11GB VRAM and 136GB DRAM, attainable on a local desktop machine, which scores even better than GPT4-0613 in BigCodeBench.

https://github.com/kvcache-ai/ktransformers

GEEKOM Mini PC Mini IT13, 13th Gen Intel i9-13900H NUC13 Mini Computers(14 Cores,20 Threads) 32GB DDR4/2TB PCIe Gen 4 SSD Windows 11 Pro Support Wi-Fi 6E/Bluetooth 5.2/USB 4.0/2.5G LAN/8K

Samsung 990 Pro SSD 4TB PCIe M.2 2280

https://docs.nvidia.com/cuda/cuda-installation-guide-linux
ChrisTitusTech says “nVidia 4070 Super with nvidia-dkms and linux-headers everything worked right away.” https://christitus.com/my-problems-with-linux

https://www.youtube.com/watch?v=Ugp6wKio1eE

https://lmstudio.ai/

https://www.reddit.com/r/LocalLLaMA/comments/15rwe7t/the_llm_gpu_buying_guide_august_2023/

https://dev.to/maximsaplin/llm-fine-tunig-on-rtx-4090-90-performance-at-55-power-29o3

https://www.hardware-corner.net/guides/hardware-for-mistral-llm/

https://www.youtube.com/watch?v=THxkY59_xko

https://www.reddit.com/r/LocalLLaMA/comments/1bhmg54/rtx_4090_vs_rtx_4500_ada_for_local_llm_training/

https://dev.to/maximsaplin/running-local-llms-cpu-vs-gpu-a-quick-speed-test-2cjn/

https://bizon-tech.com/gpu-benchmarks/NVIDIA-RTX-4090-vs-NVIDIA-RTX-4500-Ada/637vs697

ASUS ROG Maximus XII Hero version 0607 5/29/2020

https://www.amazon.com/ASUS-ROG-Z790-Motherboard-Thunderbolt/dp/B0CJMT2723

ASUS ROG Maximus Z790 Dark Hero (WiFi 7) LGA 1700(Intel 14th &13th&12th Gen) ATX Gaming Motherboard(PCIe 5.0x16, 5xM.2 Slots,DDR5,2X Thunderbolt 4 Ports, USB Type-C Front-Panel)

Brand ASUS CPU Socket LGA 1700 Compatible Devices Personal Computer RAM Memory Technology DDR5 Compatible Processors Intel Core 14th & 13th Gen Chipset Type Intel Z790

Intel LGA 1700 socket: Ready for Intel Core 13th Gen processors, Intel Core 12th Gen, Pentium Gold and Celeron Processors, 192GB RAM Robust Power Solution: 20+1+2 power stages with dual ProCool II power connectors, high-quality alloy chokes, and durable capacitors to support multi-core processors Optimized VRM Thermals: Massive heatsinks integrated with the I/O cover, joined by an L-shaped heatpipe, and connected to the power stages with high-conductivity thermal pads Latest M.2 Support: Onboard PCIe 5.0 M.2 slot, and four PCIe 4.0 M.2 slots all with substantial cooling solutions Abundant Connectivity: Dual Thunderbolt 4 USB Type-C ports, USB 20Gbps Type-C front-panel connector with Quick Charge 4+ up to 60W and USB Wattage Watcher, six additional USB 10Gbps ports, two PCIe 5.0 x16 SafeSlots, HDMI 2.1

Date First Available ‎October 17, 2023

ROG Maximus Z790 Dark Hero ASUS ROG Maximus Z790 Dark Hero (Intel 14th, 13th & 12th Gen) LGA 1700 ATX motherboard, 20+1+2 power stages, DDR5 slots, five M.2 slots, PCIe 5.0, Intel Wi-Fi 7, Thunderbolt 4 USB Type-C ports

Intel LGA 1700 socket: Ready for Intel Core 14th & 13th Gen processors, Intel Core 12th Gen, Pentium Gold and Celeron Processors Robust Power Solution: 20+1+2 power stages with dual ProCool II power connectors, high-quality alloy chokes, and durable capacitors to support multi-core processors Optimized VRM Thermals: Massive heatsinks integrated with the I/O cover, joined by an L-shaped heatpipe, and connected to the power stages with high-conductivity thermal pads Latest M.2 Support: Onboard PCIe 5.0 M.2 slot, and four PCIe 4.0 M.2 slots all with substantial cooling solutions Abundant Connectivity: Dual Thunderbolt 4 USB Type-C ports, USB 20Gbps Type-C front-panel connector with Quick Charge 4+ up to 60W and USB Wattage Watcher, six additional USB 10Gbps ports High-Performance Networking: On-board Intel Wi-Fi 7 (802.11be) and Intel 2.5 Gb Ethernet with ASUS LANGuard Intelligent Control: ASUS-exclusive AI Overclocking, AI Cooling II, AI Networking,Two-Way AI Noise Cancelation, AEMP II, and DIMM Flex to simplify setup and improve performance Immersive Audio: ROG SupremeFX ALC4082 codec with ESS ES9218 Quad DAC for up to 32-Bit/384 kHz playback

Intel® CoreTM i9-14900K New Gaming Desktop Processor 24 (8 P-cores + 16 E-cores) with Integrated Graphics - Unlocked

https://www.amazon.com/i9-14900K-Desktop-Processor-Integrated-Graphics/dp/B0CGJDKLB8

Intel Core 14th Gen processors 24 cores (8 P-cores + 16 E-cores) and 32 threads. Integrated Intel UHD Graphics 770 included Leading max clock speed of up to 6.0 GHz gives you smoother game play, higher frame rates, and rapid responsiveness Compatible with Intel 600-series (with potential BIOS update) or 700-series chipset-based motherboards