Differences

This shows you the differences between two versions of the page.

--- litespeed_wiki:cache:litemage:configuration:cache_warm_up [2016/07/22 18:11]
Rob Holda [Auto Collect List Interval (seconds)]
+++ litespeed_wiki:cache:litemage:configuration:cache_warm_up [2017/10/23 15:47] (current)
Jackson Zhang [Custom Defined URL List Files]
@@ Line 12: / Line 12: @@
     * **Store site map:** Crawl the list of pages generated directly from the database backend.
     * **Custom defined URL list:** Crawl different custom defined lists of URLs at different intervals and priorities.
-    * **Auto collected (deep crawling) URL list:** Build, maintain, and crawl a list of pages based on visitor frequency.
+    * **Auto collected (deep crawling) URL list:** Crawl a list of pages based on visitor frequency.
     * **Delta list after tag purge:** When a tag is purged, builds and crawls a list of the pages that were purged.
   * You can select and enable any combination of these options.
   * You can enable these types of warm up at the global level and disable them for certain stores; or you can disable them at global level and only enable the stores you want to crawl.
+===== LiteMage Server IP =====
+Global Setting. This setting tells the crawler where the LiteMage cache server is located so it can directly hit the server instead of having to go through DNS lookup.
+  * This setting is useful if LiteMage is located behind a CDN or proxy server or if it is located on LiteSpeed Load Balancer in front of the crawler node.
 ===== Only Run When Load Is Less Than =====
@@ Line 31: / Line 36: @@
   * Since the thread number changes based on server load, you can safely set this number higher. The recommended value is the total number of your cpu cores.
-===== Maximum Run Time (seconds) =====
+===== Maximum Run Time =====
-Global setting. Limit how long Cache Warm Up can run. Warm up cron job is set to run every 10 minutes. Set this value lower than the cron job time interval.
+Global setting. Limit how long (in seconds) Cache Warm Up can run. Warm up cron job is set to run every 10 minutes. Set this value lower than the cron job time interval.
   * The crawler will be invoked every 10 minutes. You can adjust this value in the LiteMage config.xml file (crontab:jobs:litemage_warmup_cache:schedule).
@@ Line 40: / Line 45: @@
   * If your server load is higher than the load limit, the crawler will skip one cycle or will exit earlier.
-===== Warm Up Interval (seconds) =====
+===== Warm Up Interval =====
-Set on a per store basis. Specifies how often you want to crawl the full list. Crawler request will always be served by backend directly. If you make the interval less than the public TTL, you can [[litespeed_wiki:cache:litemage:troubleshooting:warmup#keeping_public_cache_warm_indefinitely|keep your cache always warm]].
+Set on a per store basis. Specifies how often (in seconds) you want to crawl the full list. Crawler request will always be served by backend directly. If you make the interval less than the public TTL, you can [[litespeed_wiki:cache:litemage:troubleshooting:warmup#keeping_public_cache_warm_indefinitely|keep your cache always warm]].
 ===== Warm Up Priority =====
@@ Line 56: / Line 61: @@
   * If your store supports multiple currencies, you can also warm up non-default currency views. You can turn this off by leaving this setting blank.
   * You can also list only the popular currencies in the order of priority. The default currency will be crawled first, then the crawler will follow the order of your supplied list.
+===== Crawl on Behalf of Logged-in Users =====
+//Available since LiteMage version 1.3.6.//
+Warm up pages for logged-in user representatives. Space, comma separated list of user IDs from representative groups.
+For example, the user with ID 135 is from the VIP group and "Separate Cache Copy Per Customer Group" is set for the VIP group. In this case, you can list user 135 to warm the VIP user version of the site. If user 138 is in the General group and only logged-in users can write reviews and the General group is not in separate cache copy group, you can add 138, like "135, 138".
+===== Crawl for Mobile Views =====
+//Available since LiteMage version 1.3.6.//
+Warm up pages for a different mobile theme. Only use this setting if a different template theme is used for mobile view. "Mobile Theme" must also be set under LiteMage configuration - General Settings.
 ===== Custom Defined URL List Files =====
@@ Line 62: / Line 78: @@
 {{:litespeed_wiki:cache:litemage:configuration:screen_shot_2016-07-15_at_3.47.04_pm.png|}}
-Set on a per store basis. You can supply a list of files containing customized URL lists with varying intervals and priorities. This can be set by going to the “Current Configuration Scope” box and selecting a store.
+Set on a per store basis. You can supply a list of files containing customized URL lists with varying intervals and priorities. This can be set by going to the “Current Configuration Scope” box and selecting a store. The file name should an absolute path, which can be placed in magento var/litemage/, as long as the file is readable by magento php. For example, '' /home/$USER/public_html/var/litemage/cache_brand_list''. If absolute path not defined, you should see the following error:
+{{ :litespeed_wiki:cache:litemage:configuration:litemage-user-define-url-list-files-path-error.png?600 |}}
 Each line should contain the absolute path to a single file, followed by the interval for that file (in seconds) and the for that file priority. The interval must be a number greater than 600. The priority is a number greater than 0, where a lower number means a higher priority. Each file listed should contain a list of custom-defined URLs (one per line). This setting can be used to create a short list of important URLs to be warmed up first or refreshed more frequently.
-  * Absolute paths of the files containing the list of custom-defined URLs. These should contain 1 URL per line. These URLs are relative to the store baseURL (without http://domain/storepath). You can create a short list of the most important URLs, so they can be warmed up first and refreshed more frequently.
+  * Absolute paths of the files containing the list of custom-defined URLs. These should contain 1 URL per line. These URLs are relative to the store baseURL (without ''<nowiki>http://domain/storepath</nowiki>''). You can create a short list of the most important URLs, so they can be warmed up first and refreshed more frequently.
   * The file names here are an absolute path. Make sure you set the proper permissions on these files so they are readable by Magento.
   * These files can be updated whenever you want. When the crawler is invoked on each cycle, it will check if these files have changed. If they have, it will update its internal cache and restart crawling.
   * Custom Lists are set at the store level, and will follow the same rules you defined for the store, for example, what currencies to crawl.
-  * These lists only contain the relative URLs. When the crawler runs, it will add the proper base URL for that store. For example, the base URL could be something like https://www.mystore.com/en/ , but your lists should only contain the rest of the URL without its base like “accessories.html”. Use only one URL per line.
+  * These lists only contain the relative URLs. When the crawler runs, it will add the proper base URL for that store. For example, the base URL could be something like ''<nowiki>https://www.mystore.com/en/</nowiki>'' , but your lists should only contain the rest of the URL without its base like “accessories.html”. Use only one URL per line.
-===== Auto Collect List Interval (seconds) =====
+===== Auto Collect List Interval =====
 //Available since LiteMage version 1.2.0//
-Set on a per store basis. Specify how often the auto collect list is crawled.
+Set on a per store basis. Specify how often (in seconds) the auto collect list is crawled.
   * The minimum value this can be set to is half the value of the public TTL. Values that are lower than that will be ignored.
@@ Line 82: / Line 100: @@
 Set on a per store basis. The priority used when crawling the Auto Collect list. This value should be larger than 0, the lower the number, the higher the priority.
+===== Enable Auto Collect =====
+//Available since LiteMage version 1.2.0//
+//**Note:** This setting enables and disables the building and maintenance of the Auto Collect List. To enable the crawling of this list, you must also select "Auto collected (deep crawling) URL list" under the **Enable Cache Warm Up** setting.//
+//**Note:** If this setting is set to “No”, the following set of settings will be hidden from your Magento Admin Panel: Add to Auto Collect List, Include Robots in Auto Collect Visitor Count, Auto Collect Depth, Remove from Auto Collect List, and Auto Collect Time Frame (hours).//
+Enable automatically collecting additional URLs that are not currently being crawled.
+  * Once the list is fully collected, you can disable this collecting process and just crawl the collected URLs.
+  * This will also be disabled if Auto collected (deep crawling) URL list is not selected in the Enable Cache Warm Up setting.
 ===== Add to Auto Collect List =====
 //Available since LiteMage version 1.2.0//
+//**Note:** This setting is hidden unless the Enable Auto Collect setting is set to “Yes”//
 Set on a per store basis. Automatically collect additional URLs that are not currently being crawled if visited more than this number of times (excluding crawler and robots) within the amount of time defined in the Auto Collect Time Frame (hours) setting.
@@ Line 91: / Line 122: @@
   * This will also be disabled if Auto collected (deep crawling) URL list is not selected in the Enable Cache Warm Up setting.
   * This should be set to a higher number than the Remove from Auto Collect List setting.
+===== Include Robots in Auto Collect Visitor Count =====
+//Available since LiteMage version 1.2.0//
+//**Note:** This setting is hidden unless the Enable Auto Collect setting is set to “Yes”//
+Set on a per store basis. The value of this setting will tell LiteMage to include or exclude robots such as GoogleBots when determining which pages to add or remove from the Auto Collect List.
 ===== Auto Collect Depth =====
 //Available since LiteMage version 1.2.0//
+//**Note:** This setting is hidden unless the Enable Auto Collect setting is set to “Yes”//
 Set on a per store basis. The maximum number of GET parameters a URL can contain and still be auto collected.
@@ Line 101: / Line 142: @@
 //Available since LiteMage version 1.2.0//
-Set on a per store basis. Remove from Auto Collect List if a URL is visited less than this number of times (excluding the crawler and robots) within the amount of time defined in the Auto Collect Time Frame (hours).
+//**Note:** This setting is hidden unless the Enable Auto Collect setting is set to “Yes”//
+Set on a per store basis. Remove from Auto Collect List if a URL is visited less than this number of times (excluding LiteMage crawlers) within the amount of time defined in the Auto Collect Time Frame (hours).
   * This will actively keep the Auto Collect List from growing too large.
   * This should be set to a lower number than the Add to Auto Collect List setting.
+  * If this is set to 0 and Auto Collect is enabled, only 404 pages will be removed from the Auto Collect List.
-===== Auto Collect Time Frame (hours) =====
+===== Auto Collect Time Frame =====
 //Available since LiteMage version 1.2.0//
-Set on a per store basis. Auto collect counter will be reset after this time frame.
+//**Note:** This setting is hidden unless the Enable Auto Collect setting is set to “Yes”//
+Set on a per store basis. Auto collect counter will be reset after this time frame (in hours).
 ===== Delta Crawl Depth =====