This is an old revision of the document!
LiteSpeed Cache for Magento2: Crawler
The crawler travels through your site, refreshing pages that have expired in the cache. This makes it less likely that your visitors will encounter un-cached pages.
Before You Begin
- Install and enable LiteMage Cache for Magento2
- Crawler Engine: The crawler must be enabled at the server level, or you will see the warning message
Server crawler engine not enabled. Please check….. If you are using a shared hosting server, please contact your hosting provider, or see our instructions. - SiteMap: Prepare your site's sitemap, e.g.
http://magento2.com/sitemap.xml
How to Use Crawl script
Change the permissions so that the file is executable:
chmod +x cachecrawler.sh
Crawl Desktop&mobile share same theme
sh M2-crawler.sh SITE-MAP-URL
More Options
- To get help:
sh M2-crawler.sh -h - To change default interval request from 0.1s to custom NUM value:
sh M2-crawler.sh SITE-MAP-URL -i NUM
How to Generate a Sitemap
The Sitemap module is build-in for generating a sitemap in Magento 2, and it's fast.
Enable sitemap
Navigate to Magento admin page → Stores → Settings → Configuration → Catalog → XML Sitemap
Set Generation Settings Enabled to Yes
Configuring a single sitemap for all storefronts
Navigate to Magento admin page → Marketing → Seo & Search → Sitemap
- Click Add Sitemap button
- Enter value
- Filename:
sitemap.xml - Path:
/
- Click Save & Generate button
If all went well, a sitemap.xml file will generated in your magento 2 document root.
How to Verify
By using the browser developer tool, you should see X-LiteSpeed-Cache: hit,litemage at the first view
