Crawler never complete

Discussion in 'LiteSpeed Cache Plugin For WordPress' started by ForexTalk11, Jul 12, 2017.

  1. ForexTalk11

    ForexTalk11 Member

    Hi,

    The crawler never complete because of "Stopped due to reset meta position". Is that expected?

    Thanks.
     
  2. hai

    hai Member

    Hi,

    Before crawler finishes its full process, did you click "Generate Crawler File" again? That button doesn't need to be clicked manually.

    Otherwise, please check the `var` folder under your plugin, make sure all files in it is writable.
     
  3. ForexTalk11

    ForexTalk11 Member

    Hi,

    No, I don't click "Generate Crawler File". I manually run but it only craw around 1000 and then stuck there forever unless I run manually again.

    Also, Threads: 6 and cannot be increase further.

    Untitled.jpg
     
  4. hai

    hai Member

    Is your server a 6-cpu server? honestly 6 threads are high enough already.

    What is your crawler setting? `Run Duration` and `Interval Between Runs`?
     
  5. ForexTalk11

    ForexTalk11 Member

    it's 24CPU server, I just want to test how much it can handle, if it stress the server out I will change it. Unfortunately, thread is not allow update over 6.

    `Run Duration` and `Interval Between Runs` use default setting 400 and 600s.
     
  6. ForexTalk11

    ForexTalk11 Member

    @NiteWave: any imput for the issue please?

    Thanks.
     
  7. hai

    hai Member

    The reset action is triggered when there is a new position reset request, which can be only triggered when you click that button manually.
    Can you create a tix that includes your admin and ssh info so we can have a look at it?
     
  8. ForexTalk11

    ForexTalk11 Member

    As testing, I see that "reset meta position" happen when new tag added. For website which frequently have new contents and tags, the crawls will never completed.
     
  9. hai

    hai Member

    That is you have purge_all setting for add a new post.
    Every time when crawler detected a purge_all action, it will stop the crawling action and restart from the beginning.
     

Share This Page