Search results

NO, never ever! But to understand why it doesn't matter you have to know how crawling with cUrl works.

Memcached, Object cache or any other caching doesn't matter for warmup crawling. I noted no difference if it is used or not, because I am not shure if these kind of cache is available in command line interface (CLI). On this shared hosting I have PHP 7.4 and MariaDB, but it seems also not...

@Germont Have a look on attached log file and check the amount of crawled URLs and the total time to crawl 1000 URLs. Result: 1000 URLs within ~1 minute, but it can go faster depending on server ressources. This crawl session ran on a shared hosting. Server load: Low

Which feature do you need? Free Netflix access while crawling, free entry to all soccer (football) games in premier league in your country, a dozen virgins? :) I have integrated all features that are possible with cURL. Only cURL limits what is possible and impossible. Witout cURL no HighSpeed...

@Germont I invested a lot of time to develop this script. This crawler is x-time faster than everything that LiteSpeed has ever published. Before I invest time again to make a standalone version of this crawler, are you ready to pay for it? I will publish it, but not for free!!

In 2013 curl already had commands I use to make requests much faster. Possible, but this script (no plugin) is part of an application and uses sources from this application. I have to extract this sources from application to make a standalone version. It is already on my to do list, but with...

@Germont Preload ByCron crawler script is good, but he overlooks something in curl requests, that causes too much load and traffic and as result of this, requests take too much time. There are also some other deficits....

Why not? I do it and it works as fast as with my dedicated server. :)

I've developed similar working multithreading crawler, but mine is extended and has some more features for user agent, cookies, request headers for webp image, cache-control support and much more. This allows me to crawl up to 100k URLs within 1 hour on a shared hosting.

Check this: https://docs.litespeedtech.com/lsws/configuration/#logrotationsize and this: https://www.litespeedtech.com/support/wiki/doku.php/litespeed_wiki:config:real-time-stats and basically all information about Logging...

There are a lot of ways to warmup the cache, but each method does the same. It does request URLs of your page like LiteSpeed crawler does it. The question is if your provider will check this and disallows each method that crawls your page. The best way would be to kick this provider off. He...

PHP 5! WOW! You know we have year 2022?

https://www.litespeedtech.com/support/forum/threads/please-use-the-official-wordpress-forum.16028/

Is anybody interested in how to configure LSWS to use different cache directory for each (sub) domain?

FYI: I solved it. Not 100 percent perfect, but almost.

There is no good, better or bad configuration. LSWS comes as it is with the best configuration depending on your server configuration. FYI: All resources of your VPS are only virtual resources. That means 60GB Ram is VRAM only and can't be compared with RAM in a dedicated machine. The same with...

https://www.litespeedtech.com/support/forum/threads/please-use-the-official-wordpress-forum.16028/

For everyone who is interested in how to get ESI work in non Wordpress application let's start an auction. You want to know what has to be done to get ESI work? The best price will solve it. I am open for requests.

You talk about version 5.4.1 of LiteSpeed webserver? Version 6.0.11 is already released since month. https://www.litespeedtech.com/products/litespeed-web-server/release-log

https://www.litespeedtech.com/support/forum/threads/please-use-the-official-wordpress-forum.16028/