Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
litespeed_wiki:cache:lscwp:crawler [2017/11/02 15:26] Lisa Clarke |
litespeed_wiki:cache:lscwp:crawler [2018/07/05 12:14] qtwrk |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Using the LSCWP Crawler ====== | + | ====== LiteSpeed Cache for WordPress: Crawler ====== |
| The crawler travels through your site, refreshing pages that have expired in the cache. This makes it less likely that your visitors will encounter uncached pages. | The crawler travels through your site, refreshing pages that have expired in the cache. This makes it less likely that your visitors will encounter uncached pages. | ||
| Line 11: | Line 11: | ||
| {{:litespeed_wiki:cache:lscwp:lscwp-crawler-annotated.png?direct&800|}} | {{:litespeed_wiki:cache:lscwp:lscwp-crawler-annotated.png?direct&800|}} | ||
| - | - If you don't already have a sitemap file, you can generate one here by pressing the **Generate Crawler File** button. (If you already have an XML sitemap, you can enter the URL in the settings area.) | + | - If you don't already have a sitemap file, you can generate one here by pressing the **Generate Crawler File** button. (If you already have an XML sitemap, you can enter the URL on the Crawler settings tab.) |
| - Set **Activate** to ''Enable'' to turn on the crawler. | - Set **Activate** to ''Enable'' to turn on the crawler. | ||
| - If you want to watch the crawler's progress, press the **Show Crawler Status** button. | - If you want to watch the crawler's progress, press the **Show Crawler Status** button. | ||
| Line 18: | Line 18: | ||
| ===== Running the Crawler ===== | ===== Running the Crawler ===== | ||
| {{:litespeed_wiki:cache:lscwp:lscwp-crawler-watch.png?direct&800|}} | {{:litespeed_wiki:cache:lscwp:lscwp-crawler-watch.png?direct&800|}} | ||
| + | |||
| + | |||
| If you've opted to watch the crawler status, your screen will look something like this. The messages in the status window will vary from these, as this screenshot was grabbed from a small installation with few pages to crawl. | If you've opted to watch the crawler status, your screen will look something like this. The messages in the status window will vary from these, as this screenshot was grabbed from a small installation with few pages to crawl. | ||
| + | |||
| + | {{:litespeed_wiki:cache:lscwp:troubleshooting:lscwp-crawler2.jpg|}} | ||
| + | |||
| + | This is an explanation: | ||
| + | |||
| + | ''Size: 181'': This means the sitemap files has 181 URLs, one URL per line. | ||
| + | |||
| + | ''Crawler: #1'': This means you are watching action of Crawler number 1 , there could be multiple crawlers working depends on your setting. | ||
| + | |||
| + | ''Position: 1'': This means this crawler is currently fetching the 1st URL from sitemap file. | ||
| + | |||
| + | ''Threads: 2'': This means this is thread number 2, there could be multiple threads fetching, it is smart and will adjust based on your load [[litespeed_wiki:cache:lscwp:configuration:crawler|settings]]. | ||
| + | |||
| + | ''Status: Stopped due to reset meta position'': This means while it's crawling, site has purged or sitemap changed, so it will restart from top. | ||
| + | |||
| + | |||
| + | |||
| If you wish to keep a particular path from being crawled, you may enter it in the **Sitemap Generation Blacklist** box and press **Save**. After the crawler has run for the first time, if it encounters any pages marked ''do-not-cache'' they will be added to this Blacklist automatically. | If you wish to keep a particular path from being crawled, you may enter it in the **Sitemap Generation Blacklist** box and press **Save**. After the crawler has run for the first time, if it encounters any pages marked ''do-not-cache'' they will be added to this Blacklist automatically. | ||