Differences

This shows you the differences between two versions of the page.

--- litespeed_wiki:cache:lscwp:configuration:enabling_the_crawler [2018/12/13 21:56]
Jackson Zhang [On a LiteSpeed Native environment]
+++ litespeed_wiki:cache:lscwp:configuration:enabling_the_crawler [2019/08/02 18:44]
Jackson Zhang [Testing]
@@ Line 136: / Line 136: @@
 </code>
-===== On a LiteSpeed Native environment =====
+==== ''CacheEngine -crawler'' ====
+Starting from LSWS 5.3.5 or later, in any situation, if you just want to ensure to disable crawler for apache virtual host, you can add ''CacheEngine -crawler'' to the Apache virtual host configuration.
+  <IfModule LiteSpeed>
+  CacheEngine -crawler
+  </IfModule>
-Crawler is enabled by default on an LSWS native environment.
+''CacheEngine -crawler''(this is supported in LSWS v5.3.5 and later) in
-To disable it from Server Level, in the LSWS WebAdmin interface, navigate to **LSWS Admin > Server > General > Apache Style Configurations**, click **Edit**, and make sure it reads ''CacheEngine -crawler''(this is supported from LSWS 5.3.5).
-To disable it from Virtual Host level, you can add the vhost level Apache-style configuration ''CacheEngine -crawler'' to turn off the crawler.
+===== In a LiteSpeed Native Environment =====
+The cache crawler is enabled by default in a LSWS native environment.
+To disable it at the Server Level, in the LSWS WebAdmin interface, navigate to **LSWS Admin > Configuration > Server > Cache > ** Make sure ''Crawler'' is unchecked, and ''On''and ''ESI'' is checked. If ''Not Set'' is checked here, it will use the default value to turn ALL ''on'' including ''Crawler''.
+{{:litespeed_wiki:cache:lscwp:configuration:disable-crawler-lsws-native-1.png?600|}}
+To disable the cache crawler at the  LSWS native Virtual Host level, you can go to **LSWS Admin > Configuration > Virtual Host > VH Name > Cache >** Check ''on'' and ''ESI'' and uncheck ''Crawler''. If ''Not Set'', it will inherit the server-level configuration.
+Please note that don't turn on ''Enable LiteMage'' since this setting will enable Crawler even ''Crawler'' is unchecked.
+{{:litespeed_wiki:cache:lscwp:configuration:disable-crawler-lsws-native-vh-1.png?600|}}
 To add any of the optional server variables, navigate to **Server > External App** and add the variable(s) to the **Environment** setting, one per line. For example:
@@ Line 151: / Line 167: @@
 {{:litespeed_wiki:cache:lscwp:configuration:lscwp-admin-crawler.png?600|}}
+===== Testing =====
+LiteSpeed Web server cache engine will set envirment varibles for ''X-LSCACHE''. You can always check Envirment Variables through phpinfo page to see if crawler is on or not. If the crawler is not there, then it has been disabled successfully. LSWS can only disable LiteSpeed cache plugin or LiteSpeed crawler since such LiteSpeed crawlers will check ''X_LSCACHE'' environment variable. LSWS can not stop any third party crawler from working since they don't check ''X_LSCACHE'' to act accordingly.
+  $_SERVER['X-LSCACHE']	on,esi
+{{:litespeed_wiki:cache:lscwp:configuration:disable-crawler-lsws-native-phpinfo-1.png?600|}}
+On LiteSpeed cache for WordPress plugin, under ''crawler'', it should show ''Crawler Cron'' ''Disable'' and Warning ''The crawler feature is not enabled on the LiteSpeed server. Please consult your server admin.''.
+{{:litespeed_wiki:cache:lscwp:configuration:disable-crawler-lsws-native-wp-plugin-cron-status-1.png?800|}}