Differences

This shows you the differences between two versions of the page.

--- litespeed_wiki:cache:lscwp:configuration:enabling_the_crawler [2018/12/13 22:15]
Jackson Zhang [On a LiteSpeed Native environment]
+++ litespeed_wiki:cache:lscwp:configuration:enabling_the_crawler [2019/08/05 15:07]
Jackson Zhang
@@ Line 136: / Line 136: @@
 </code>
-===== On a LiteSpeed Native environment =====
+==== ''CacheEngine -crawler'' ====
+Starting from LSWS 5.3.5 or later, in any situation, if you just want to ensure to disable crawler for apache virtual host, you can add ''CacheEngine -crawler'' to the Apache virtual host configuration.
+  <IfModule LiteSpeed>
+  CacheEngine -crawler
+  </IfModule>
-Crawler is enabled by default on an LSWS native environment.
+''CacheEngine -crawler''(this is supported in LSWS v5.3.5 and later) in
-To disable it from Server Level, in the LSWS WebAdmin interface, navigate to **LSWS Admin > Configuration > Server > General > Apache Style Configurations**, click **Edit**, and make sure it reads ''CacheEngine -crawler''(this is supported from LSWS 5.3.5).
-{{ :litespeed_wiki:cache:lscwp:configuration:lsws-native-crawler-disable-1.png?800 |}}
+===== In a LiteSpeed Native Environment =====
-{{ :litespeed_wiki:cache:lscwp:configuration:lsws-native-crawler-disable-2.png?800 |}}
+The cache crawler is enabled by default in a LSWS native environment.
-To disable it from Virtual Host level, you can add the vhost level Apache-style configuration ''CacheEngine -crawler'' to turn off the crawler.
+To disable it at the Server Level, you will need to use LSWS 5.4 and above version, since there is a new **Cache Features** function added to control this.
-{{ :litespeed_wiki:cache:lscwp:configuration:lsws-native-crawler-disable-vh-3.png?800 |}}
+In the LSWS WebAdmin interface, navigate to **LSWS Admin > Configuration > Server > Cache**. In **Cache Features**, check ''On'', uncheck ''Crawler'', check ''ESI'', and uncheck ''Not Set''.
+If ''Not Set'' is checked, the other three values will be ignored and the default values will be used. (By default, all three are checked.)
+{{:litespeed_wiki:cache:lscwp:configuration:disable-crawler-lsws-native-1.png?600|}}
+To disable the cache crawler at the  LSWS native Virtual Host level, you can go to **LSWS Admin > Configuration > Virtual Host > VH Name > Cache >**, and set **Cache Features** in the same manner as above. If ''Not Set'' is checked, the other three values will be ignored and the server-level configuration will be inherited.
+Please note: Do not set **Enable LiteMage** to ''On'', as this setting will also enable the crawler, even if ''Crawler'' is unchecked.
+{{:litespeed_wiki:cache:lscwp:configuration:disable-crawler-lsws-native-vh-1.png?600|}}
 To add any of the optional server variables, navigate to **Server > External App** and add the variable(s) to the **Environment** setting, one per line. For example:
@@ Line 156: / Line 170: @@
 {{:litespeed_wiki:cache:lscwp:configuration:lscwp-admin-crawler.png?600|}}
+===== Testing =====
+LiteSpeed Web server cache engine will set environment varibles for ''X-LSCACHE''. You can always check Envirment Variables through phpinfo page to see if crawler is on or not. If the crawler is not there, then it has been disabled successfully. LSWS can only disable the LiteSpeed cache plugin or LiteSpeed crawler since such LiteSpeed crawlers will check ''X_LSCACHE'' environment variable. LSWS can not stop any third party crawler from working since they don't check ''X_LSCACHE'' to act accordingly.
+  $_SERVER['X-LSCACHE']	on,esi
+{{:litespeed_wiki:cache:lscwp:configuration:disable-crawler-lsws-native-phpinfo-1.png?600|}}
+In the LiteSpeed cache for WordPress plugin, under **Settings > Crawler**, it should show **Crawler Cron** set to ''Disable'', and
+  Warning: The crawler feature is not enabled on the LiteSpeed server. Please consult your server admin.
+{{:litespeed_wiki:cache:lscwp:configuration:disable-crawler-lsws-native-wp-plugin-cron-status-1.png?800|}}