Swapping

Discussion in 'General' started by Giorgi, Oct 16, 2007.

  1. Giorgi

    Giorgi New Member

    Hi there

    I have some questions about swapping...

    I am running 3gb RAM/Fedora 6/Core duo

    1. I wonder what is: Max I/O Buffer Size is this RAM memory that litespeed can use to handle big files? So, if the limit is reached then litespeed will begin to dump in TMP directory and use it as RAM I guess?

    2. What will later clean Swapping Directory ? how can I speed up cleaning TMP directory and is it recommended?

    3. My server must handle large file uploads (650mb) what would you recommend? should I increase Max I/O Buffer Size or increase TMP directory size?

    Thank you
  2. mistwang

    mistwang LiteSpeed Staff

    Yes,
    You do not need to. It should not have big impact on performance, and LSWS will avoid using it whenever possible.

    Handling upload of 650mb file via HTTP then processing with PHP is not recommended. It is not efficient with too much overhead, better use FTP for file that big. If you have a few concurrent uploads, your server might be overloaded.
  3. mistwang

    mistwang LiteSpeed Staff

    In theory, larger I/O buffer should be better, but I think it does not matter much what parameter you tune for such large file. TMP directory size will grow automatically as long as there are enough free space.
  4. Giorgi

    Giorgi New Member

    Hm...

    I have big problem, you see I am running Lsws enterprise on my file hosting site, it must accept that kind of files (650mb) upload via flash, but after about 1 hour of uploading I get IO error with filename

    in my admin panel there is:

    Code:
    2007-10-16 22:31:10.800 ERROR [89.232.19.40:1509-0#APVH_www.mysite.com] out of swapping space while reading request body! 
    so what do you advice me to do?
  5. mistwang

    mistwang LiteSpeed Staff

    Please try setting "Max I/O buffer size" to "0", yes "zero", the swapping space will be limited by available disk space under /tmp.
    Performance wise, you may not notice big difference unless the disk is too busy, server is disk I/O bound.
    Use a dedicate disk for /tmp/lshttpd will help.
  6. Giorgi

    Giorgi New Member


    So I should consider increasing tmp partition?
  7. mistwang

    mistwang LiteSpeed Staff

  8. Giorgi

    Giorgi New Member

    Now I am confused...

    1. I thought that error I posted was caused because that I dont had enough space on tmp partition... is not that right?

    2. and I have 3gb RAM, why litespeed wont use it at first place and then start to swap to tmp?
  9. mistwang

    mistwang LiteSpeed Staff

    No, when buffer memory was used up in the middle of a request, this error will occur. LSWS cannot switch from buffer memory to swap memory in the middle of a request. Some area can be improved in the future.

    If LSWS uses all 3gb RAM for the buffer, the server will become extremely slow. So, there is a configurable limit. once the limit is hit, it will start swap to tmp for new requests.

    I think we will avoid buffer large request body in the buffer memory, instead, it will go straight to the tmp swap.

    Even when swap is used, it is more likely that the swap file is cached in the memory, so the performance hit is minimum, and the OS kernel has more freedom to use system memory efficiently.
  10. Giorgi

    Giorgi New Member

    so I will need to increase tmp partition... I did so actually... increased it up to 10gb and progress went up to 83% then IO error occured again (before it only went up to 50%)

    I dont understand why more than 10gb of tmp is needed to upload just 650mb file...

    maybe something else is generating this error...

    can Connection Timeout (secs) trigger that? I have it at 10 000 seconds... (650mb needs about 3.50 hours to upload)

    I guess 10 000 seconds is hard limit, is there anyway to increase it further?
  11. ts77

    ts77 New Member

    did you try what mistwang suggested above?
  12. Giorgi

    Giorgi New Member

    Yes of course, I am currently have it to zero and now perfomance is down, server waits about 5-6 seconds before page load and it still only uploads up to 500mb and thats all...

    I dont understand, I thought Lsws was capable to handle large uploads and loads, thats why I moved to enterprise license, I am dissapointed but I really dont want to switch back to Apache.

    is there any hope for me?
  13. mistwang

    mistwang LiteSpeed Staff

  14. Giorgi

    Giorgi New Member

    Yes, it servers downloads too, but what it has to do with uploads? I am not php programmer, its just php script which uploads files and generates download links for it.

    thats all
  15. mistwang

    mistwang LiteSpeed Staff

    Does the download link point to a PHP script or directly to the static file?

    If static file is streamed through PHP script, the static file will be buffered by LSWS as well, so it is not efficient at all.
  16. ts77

    ts77 New Member

    ... and that has nothing to do with your upload problems, its "just" a "related note" ;).
  17. Giorgi

    Giorgi New Member

    Yes its streamed over php I guess, since my static file is kind of masked in these kind of link:

    Code:
    http://mysite.com/index.php?p=download&hash=tpYMEw24zUY3
  18. mistwang

    mistwang LiteSpeed Staff

    That's why you run out of buffer/swap space. LSWS caches the whole response body produced by PHP, so all download files will be stored in the buffer/swap.

    You should have your developer to remove the streaming code and replace it with a internal redirect I mentioned earlier. Your site will be flying after that. The large upload might not be a problem any more.
  19. Giorgi

    Giorgi New Member

    Unfortunatelly its impossible because of nature of the script :(

    I guess there is no other way, either I should switch back to apache or increase TMP partition again :(
  20. mistwang

    mistwang LiteSpeed Staff

    We will try to have your issue addressed in up coming 3.3 release.
    It may not be the issue of your TMP partition. If you run lshttpd in 32bit system, you may run into virtual memory limit problem if there a few upload/download session for large files. 3.3 release will have it addressed. It is due in next week.

Share This Page