This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
litespeed_wiki:cache:lscwp:crawler [2017/11/02 15:40]
Lisa Clarke [Getting Ready to Run]
litespeed_wiki:cache:lscwp:crawler [2020/11/14 15:32]
Lisa Clarke Redirect to new Documentation Site
Line 1: Line 1:
-====== Using the LSCWP Crawler ====== +~~REDIRECT>​https://docs.litespeedtech.com/​lscache/​lscwp/crawler/~~
- +
-The crawler travels through your site, refreshing pages that have expired in the cache. This makes it less likely that your visitors will encounter uncached pages. +
- +
-===== Before you Begin ===== +
-The crawler must be enabled at the server level, or it will not work. Please contact your hosting provider, or see [[litespeed_wiki:cache:​lscwp:​configuration:​enabling_the_crawler|our instructions]]. +
- +
-Before using any of the functions in this area, please be sure that you have [[litespeed_wiki:​cache:​lscwp:​configuration:​crawler|configured the crawler]] to your liking. +
- +
-===== Getting Ready to Run ===== +
-{{:​litespeed_wiki:​cache:​lscwp:lscwp-crawler-annotated.png?​direct&​800|}} +
- +
-  - If you don't already have a sitemap file, you can generate one here by pressing the **Generate Crawler File** button. (If you already have an XML sitemap, you can enter the URL on the Crawler settings tab.) +
-  - Set **Activate** to ''​Enable''​ to turn on the crawler. +
-  - If you want to watch the crawler'​s progress, press the **Show Crawler Status** button. +
-  - You can wait for the crawler to begin on its own according to its configuration settings, or you can press the **Manually Run** button and start it immediately. If the crawler is already running, this button will be ignored. +
- +
-===== Running the Crawler ===== +
-{{:​litespeed_wiki:​cache:​lscwp:​lscwp-crawler-watch.png?​direct&​800|}} +
- +
-If you've opted to watch the crawler status, your screen will look something like this. The messages in the status window will vary from these, as this screenshot was grabbed from a small installation with few pages to crawl. +
- +
-If you wish to keep a particular path from being crawled, you may enter it in the **Sitemap Generation Blacklist** box and press **Save**. After the crawler has run for the first time, if it encounters any pages marked ''​do-not-cache''​ they will be added to this Blacklist automatically. +
  • Admin
  • Last modified: 2020/11/14 15:32
  • by Lisa Clarke