Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
|
litespeed_wiki:cache:no-plugin-crawler [2019/04/17 14:27] Jackson Zhang created |
litespeed_wiki:cache:no-plugin-crawler [2020/09/17 20:13] Lisa Clarke Redirect to new Documentation Site |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Can I setup crawler for rewrite rule based cache? ====== | + | ~~REDIRECT>https://docs.litespeedtech.com/lscache/noplugin/settings/~~ |
| - | You may wonder: "Can I setup crawler for rewrite ruble-based cache? Do you have such crawler solution for it if I have a sitemap for my customized application? | + | |
| - | + | ||
| - | Normally there is no need to setup crawler for rewrite rule based cache. The typical case for rewrite rule-based cache setup is to set cache for a very short time, such as 2 minutes. Then cache files will be expired and the site will need to re-cache again. The reason to set it to a very short time is to handle the cache purge. If you set it for a long time, any changes to that page won't be updated until the cache expired. 2 minutes could be a good timeframe meets your tolerate for the updated page not to be refreshed. Of cause, you can set that cache TTL timeframe to be longer whichever you can are tolerant. | + | |
| - | + | ||
| - | In some rare case, such as a user sets the cache TTL to 30 mins in the rewrite rules and still want the site to be crawled. The user has sitemap already for the site. How to enable crawler function within LSWS and is there any universal crawler script available? The answer is yes. | + | |
| - | + | ||
| - | ===== Enable "crawler" on LiteSpeed server (Apache configuration) ===== | + | |
| - | Firstly, you will need to enable "crawler" for your LiteSpeed web server since "crawler is normally disabled by default. In control panel environment or LiteSpeed reads apache configuration, you should set the following at apache configuration. | + | |
| - | <IfModule Litespeed> | + | |
| - | CacheEngine on, crawler | + | |
| - | </IfModule> | + | |
| - | + | ||
| - | Please refer the detailed steps [[litespeed_wiki:cache:common_installation:shared-enable-individually|here]]. | + | |
| - | + | ||
| - | ===== Try a Magento 2 shell script crawler when you have a sitemap ===== | + | |
| - | Secondly, you can use any third party or your own crawler script to warm up your customized application. We do have a Magento 2 shell script crawler available as far as you have a sitemap for your site. Please refer [[https://www.litespeedtech.com/support/wiki/doku.php/litespeed_wiki:cache:litemage2:crawler|here]] for details. | + | |
| - | + | ||
| - | ===== Troubleshooting ===== | + | |
| - | ==== Try the above script but not working? ==== | + | |
| - | Most likely you did not set ''CacheEngine on, crawler'' in apache configuration. Set it if you have missed it. | + | |
| - | + | ||
| - | The server may have a domain override at ''/etc/hosts'' like ''127.0.0.1 yourdomain.com www.yourdomain.com''. It will break the crawler. Commenting it should solve the issue. | + | |
| - | + | ||
| - | If still an issue, you can log a ticket with us to have further investigations. | + | |
| - | + | ||
| - | | + | |