Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Next revision Both sides next revision | ||
|
litespeed_wiki:cache:litemage2:crawler [2019/01/17 21:46] Eric Leu |
litespeed_wiki:cache:litemage2:crawler [2019/04/04 15:56] Eric Leu [More Options] |
||
|---|---|---|---|
| Line 16: | Line 16: | ||
| * To get help: ''bash M2-crawler.sh -h'' | * To get help: ''bash M2-crawler.sh -h'' | ||
| * To change default interval request from 0.1s to custom NUM value: ''bash M2-crawler.sh SITE-MAP-URL -i NUM'' | * To change default interval request from 0.1s to custom NUM value: ''bash M2-crawler.sh SITE-MAP-URL -i NUM'' | ||
| + | * To crawl with cookie set: ''bash M2-crawler.sh -c SITE-MAP-URL'' | ||
| + | * To store log in /tmp/M2-crawler.log: ''bash M2-crawler.sh -v SITE-MAP-URL'' | ||
| + | * To debug one URL and output on screen: ''bash M2-crawler.sh -d SITE-URL'' | ||
| ===== How to Generate a Sitemap===== | ===== How to Generate a Sitemap===== | ||