Magic curl option for fast recache and small lscache size

serpent_driver

Well-Known Member
#45
Don't think complicate, use it and enjoy to save data that has not to be unnecessary downloaded. All, but not LScache extension for OC use this method.
 
Last edited:

AndreyPopov

Well-Known Member
#46
some statistics about "magic" option

same!!!! server load - max three core(three processes) from four

recache time:
- without "magic" option full recache for 3(three) UAs for 9000 products: more than 3 days
- with "magic" option full recache for 12(twelve) UAs for 9000 products: less than 24 hours

cache storage:
- without "magic" option full recache for 5(five) UAs for 9000 products: 60GB
- with "magic" option full recache for 18(eighteen) UAs for 9000 products: 22 GB
 

serpent_driver

Well-Known Member
#49
To check the load of a server there is a common technic from Linux. Checking the number of used processes gives no information about the load. You can simple check the load if you run "uptime" in command line interface. The result you get displayed gives you more information about the load, but on shared hosting it is difficult to measure the load, because you are not the only one who causes load.

If you don't have CLI access you can also get information about the load with sys_getloadavg() PHP function. Use the example code of php.net how to get this information: https://www.php.net/manual/en/function.sys-getloadavg.php

FYI: Your Hosting also uses this method.
 
Last edited:

serpent_driver

Well-Known Member
#51
some statistics about "magic" option
cache storage:
- without "magic" option full recache for 5(five) UAs for 9000 products: 60GB
- with "magic" option full recache for 18(eighteen) UAs for 9000 products: 22 GB
To explain what seems to be "magic" related to the disk space, here are some additional information. First of all what @AndreyPopov found out is not magic. LiteSpeed only forgot to add a default curl option in code. This option has affect on how cache files are stored. If this option is set it will compress the cache files and as everybody knows compressing will reduce the file and the disk space. This simple public option reduces the disk space dramatically.
 

serpent_driver

Well-Known Member
#54
for LSCache for Opencart - this is magic!
Yes, Mr. Magic :) reduced on this it is maaaaaaagic! :) You've earned an award for it

why?
I have 4 core in exclusive use according to my tariff plan.
Because nobody can accept it for his own use. Your load is related on your server but each server has different ressources. That is why Linux offers a method to check it really, but always related to the used server with different power.
 

AndreyPopov

Well-Known Member
#56
Your load is related on your server but each server has different ressources.
I compare on:
- same server
- same configuration
- same conditions


24 hours for 9000 URLs? No problem, I do it in 1 hour with less load.
you are really stupid man :(
hear only yourself.
never read what others wrote.

- 9000 product links + 9000 Journal QuickView popup links + 9000 Journal option popup links = 27000 links
- 12 (twelve) UAs: 27000 * 12 = 324000 links (13500 links per hour)

again and again: open your eyes, free your mind and break edges of your knowledges
 

AndreyPopov

Well-Known Member
#58
YI just crawled your page within 1 hour., 16.000 Urls with 11 different cache profiles.
you crawled already crawled(cached) pages? really?!?!?!?!?!?!?!? and proud? :eek:

open your eyes, free your mind and break edges of your knowledges

P.S. another two fixes (after my tickets) that LS "forgot"
https://github.com/litespeedtech/lscache-opencart/commit/d96d45cc814aca8cdc009bd9086345ae263db4c4
https://github.com/litespeedtech/lscache-opencart/commit/cd1c069b42502c4dc39dba8c8d6b7fd5d8a66155
 
Last edited:

serpent_driver

Well-Known Member
#59
As long you don't acceppt other advices you will never be an expert for caching and for improvements for faster cache wamup. For your information I developed a lot of applications for LScache and for warming the cache. In comparison to me your are a Newbie without knowing what he is doing.
 

AndreyPopov

Well-Known Member
#60
For your information I developed a lot of applications for LScache and for warming the cache.
this is NOTHING matter.
because you hear only yourself.
all our discussions here you only can say: you are wrong, you are wrong.
but ...... LS developers have another point of view and ADD all my ideas and founded problems to code.
and all your "help" only bla-bla-bla-bla: I'm great crawler developer :p

In comparison to me your are a Newbie without knowing what he is doing.
but I try find and solve problems and enhance features of LSCache plugin for Opencart
and you only bla-bla-bla-bla: I'm great crawler developer :p
 
Top