_Cagkan_Sayin_Alamy.jpg?#)
![[The AI Show Episode 139]: The Government Knows AGI Is Coming, Superintelligence Strategy, OpenAI’s $20,000 Per Month Agents & Top 100 Gen AI Apps](https://www.marketingaiinstitute.com/hubfs/ep%20139%20cover-2.png)
![[The AI Show Episode 138]: Introducing GPT-4.5, Claude 3.7 Sonnet, Alexa+, Deep Research Now in ChatGPT Plus & How AI Is Disrupting Writing](https://www.marketingaiinstitute.com/hubfs/ep%20138%20cover.png)
![Release: Rendering Ranger: R² [Rewind]](https://images-3.gog-statics.com/48a9164e1467b7da3bb4ce148b93c2f92cac99bdaa9f96b00268427e797fc455.jpg)
.jpg?width=1920&height=1920&fit=bounds&quality=80&format=jpg&auto=webp#){kind=tracking}
Free Local RAG Scraper for Custom GPTs and Assistants
TL;DR Scrape Squarespace/Shopify Websites for use with AI This web scraper runs entirely in your browser and is perfect for creating training data for AI models. It works by reading the website's sitemap.xml file, making it particularly well-suited for modern platforms like Squarespace and Shopify that automatically generate sitemaps.The scraper preserves the structure of your content, including headings, paragraphs, lists, and tables, while removing unnecessary elements like navigation menus and footers. It also captures metadata, images, and PDF documents. Technical Details This scraper uses a CORS proxy to access websites. No additional setup is required.The scraper will:Read the website's sitemap.xml to find all pages. It runs in your browser and none of your information is stored on a server.Process each page while preserving content structure.Generate a markdown file with all content. You can add this file as knowledge to a custom GPT or attach it to an AI assistant.Allow you to preview each page's content before saving. Start Scraping Save Markdown Ready to start Select Page: No pages scraped yet Cursor AI did most of the heavy lifting in developing this app.If you have improvement suggestions, please share them in the comments.
TL;DR Scrape Squarespace/Shopify Websites for use with AI
This web scraper runs entirely in your browser and is perfect for creating training data for AI models. It works by reading the website's sitemap.xml file, making it particularly well-suited for modern platforms like Squarespace and Shopify that automatically generate sitemaps.
The scraper preserves the structure of your content, including headings, paragraphs, lists, and tables, while removing unnecessary elements like navigation menus and footers. It also captures metadata, images, and PDF documents.
-
This scraper uses a CORS proxy to access websites. No additional setup is required.
The scraper will:
Read the website's sitemap.xml to find all pages. It runs in your browser and none of your information is stored on a server.
Process each page while preserving content structure.
Generate a markdown file with all content. You can add this file as knowledge to a custom GPT or attach it to an AI assistant.
Allow you to preview each page's content before saving.
Cursor AI did most of the heavy lifting in developing this app.
If you have improvement suggestions, please share them in the comments.
