SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 2 months agoWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?message-squaremessage-square4linkfedilinkarrow-up10arrow-down10file-text
arrow-up10arrow-down1message-squareWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 2 months agomessage-square4linkfedilinkfile-text
minus-squareplutopos@lemmy.ziplinkfedilinkarrow-up1·1 month agoThere are no lower or upper bounds for local LLMs. Just get the best GPU/RAM combo you can with your budget. Some laptops also have NPUs that are dedicated specifically to AI iirc
minus-squareSocialistVibes01@lemmy.mlOPlinkfedilinkarrow-up1·28 days agoThanks. What about NVidia vs AMD? Should I go with AMD if cheaper?
minus-squareplutopos@lemmy.ziplinkfedilinkarrow-up1·27 days agoAMD has better Linux support if you plan on using it
There are no lower or upper bounds for local LLMs. Just get the best GPU/RAM combo you can with your budget. Some laptops also have NPUs that are dedicated specifically to AI iirc
Thanks. What about NVidia vs AMD? Should I go with AMD if cheaper?
AMD has better Linux support if you plan on using it