![iFixit Tears Down New M4 MacBook Air [Video]](https://www.iclarified.com/images/news/96717/96717/96717-640.jpg)
![Apple Officially Announces Return of 'Ted Lasso' for Fourth Season [Video]](https://www.iclarified.com/images/news/96710/96710/96710-640.jpg)
![9to5Rewards: Last chance to win a MacBook Pro from Chargeasap [Giveaway]](https://i0.wp.com/9to5mac.com/wp-content/uploads/sites/6/2024/10/M4-MacBook-Pro-doesnt-tempt-me-because-Apple-Silicon-Macs-are-almost-too-good.jpg?resize=1200%2C628&quality=82&strip=all&ssl=1)
![[The AI Show Episode 139]: The Government Knows AGI Is Coming, Superintelligence Strategy, OpenAI’s $20,000 Per Month Agents & Top 100 Gen AI Apps](https://www.marketingaiinstitute.com/hubfs/ep%20139%20cover-2.png)
![[The AI Show Episode 138]: Introducing GPT-4.5, Claude 3.7 Sonnet, Alexa+, Deep Research Now in ChatGPT Plus & How AI Is Disrupting Writing](https://www.marketingaiinstitute.com/hubfs/ep%20138%20cover.png)
![How to become a self-taught developer while supporting a family [Podcast #164]](https://cdn.hashnode.com/res/hashnode/image/upload/v1741989957776/7e938ad4-f691-4c9e-8c6b-dc26da7767e1.png?#)
![[FREE EBOOKS] ChatGPT Prompts Book – Precision Prompts, Priming, Training & AI Writing Techniques for Mortals & Five More Best Selling Titles](https://www.javacodegeeks.com/wp-content/uploads/2012/12/jcg-logo.jpg){kind=logo}
.jpg?#)
.jpg?#)
Best architecture to use LLM on laptop app [closed]
I'm designing an App that takes a document and then gets an LLM to review it. Ideally it would: run on a low end laptop (ie no GPU), work without internet access, run with minimal cost and have a RAG system that informs the review. I realise I might have to compromise on some of these. I've thought of three architectures: A. Make remote calls to eg Chatgpt with RAG on the laptop. This presumably incurs API call costs. B. Run RAG+eg mistral on the local machine. This might need a more powerful machine which I don't want. C. Run the whole thing in the cloud and just make calls to it from the laptop. Maybe use Chatgpt, maybe use llama or similar. Obviously cloud costs here. Is there a better/standard way?
![Best architecture to use LLM on laptop app [closed]](https://cdn.sstatic.net/Sites/softwareengineering/Img/apple-touch-icon@2.png?v=1ef7363febba)
I'm designing an App that takes a document and then gets an LLM to review it. Ideally it would:
- run on a low end laptop (ie no GPU),
- work without internet access,
- run with minimal cost and
- have a RAG system that informs the review.
I realise I might have to compromise on some of these.
I've thought of three architectures:
A. Make remote calls to eg Chatgpt with RAG on the laptop. This presumably incurs API call costs.
B. Run RAG+eg mistral on the local machine. This might need a more powerful machine which I don't want.
C. Run the whole thing in the cloud and just make calls to it from the laptop. Maybe use Chatgpt, maybe use llama or similar. Obviously cloud costs here.
Is there a better/standard way?
