Swap Folder Cleanup

Discussion in 'Feedback/Feature Requests' started by Tony, Jun 4, 2010.

  1. Tony

    Tony Member

    I'd really like to see some sort of cleanup of the swap folder. It stores a few things in there so I'll address these in separate points.

    1) The gzip files data stored should have some sort of cleanup of old files mechanism. It seems like it just grows over time so it hits 500MB of gzip files then 2 weeks it's 1GB. I'm sure we could put it in a partition with 100GB and eventually we'd fill it. So some sort of cleaning up mechanism that is used to clean up files not used or written x amount of time ago. I'd obviously say based on access time but not sure if that data would be available to LSWS or not or how it handled it.


    2) Cleaning up of partially uploaded files. This does not seem to be happening. So over time it can grow quite large due to this. So I'd like to have it say remove attempted uploaded files after a period of time and there is no connection using it. There is no point in having a bunch of 20MB uploaded files that are not used by anything.



    Right now we wipe the swap folder every few weeks but I don't think that's a great solution. We'd rather see LSWS pro actively cleaning up unused files in this folder. There is no point in keeping gzipped files that were last accessed a week and a half ago. There is also no point in keeping partially uploaded files either.
  2. ffeingol

    ffeingol Member

    I'll second this one.
  3. mistwang

    mistwang LiteSpeed Staff

    OK, we will do something about it.
    Actually, those files can be safely removed with "cd swap_dir; rm -rf *".
  4. ffeingol

    ffeingol Member

    So basically we can just rm -rf all the folders under the swap dir and LSWS will just recreate the structure? Does LSWS need to be down when we do that?
  5. mistwang

    mistwang LiteSpeed Staff

    that's correct, not need to bring down LSWS.
  6. Tony

    Tony Member


    Looks like it's part of 4.0.16. Any more details on what exactly the cleanup does?
  7. mistwang

    mistwang LiteSpeed Staff

    It calls lsws/admin/misc/cleancache.sh once a day. The behavior can be customized via the shell script. Default is to remove file more than one day old based on modified time. Can be changed to based on last access time, but some system may use "noatime" option when mount the partition.
  8. ffeingol

    ffeingol Member

    How does it get called (cron etc.) because it's doing nothing on at least one server.
  9. mistwang

    mistwang LiteSpeed Staff

    It was called by LSWS server daemon when server check for new update. So, you should let LSWS check for update daily instead of monthly in order for daily cleanup.
  10. ffeingol

    ffeingol Member

    check for updates is already set to daily. Is the execution logged anywhere so we can see why it's not working (or maybe not even running)?
  11. mistwang

    mistwang LiteSpeed Staff

    It should have a log message at NOTICE level
    "Start to clean up cache directory."
  12. mistwang

    mistwang LiteSpeed Staff

    You can try running "lsws/admin/misc/cleancache.sh <cache_dir>" see if it does anything for you.
  13. ffeingol

    ffeingol Member

    OK, I can see the messages that it's starting but it's doing nothing. In looking at the script it looks like it's looking for swap/0/file and our directory is swap/0/0/file. Is this just old junk from upgrades?
  14. NiteWave

    NiteWave Administrator

    "it's doing nothing" -- may due to a typo in the script: root_dr should be root_dir
  15. ffeingol

    ffeingol Member

    OK, I fixed the script (there are actually 2 occurrences of the typo). Will there be a new release that fixes this typo?
  16. ffeingol

    ffeingol Member

    Still does not appear to work. If I'm reading the script correctly, it's looking for files in:

    /cache/0
    /cache/1

    etc

    but the cache directory is actually organized

    /cache/0
    /cache/0/0
    /cache/0/1
    /cache/1
    /cache/1/0
    /cache/1/1

    Can anyone else confirm if this is working or not working?
  17. anewday

    anewday Moderator

    I see this in the error_log right after I upgraded to 4.0.16
    Code:
    Start to clean up cache directory.
    2010-08-08 21:38:49.293 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.300 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.301 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.303 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.304 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.306 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.307 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.309 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.310 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.312 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.313 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.315 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.316 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.318 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.319 [STDERR] find: invalid predicate `-delete'
    2010-08-08 21:38:49.321 [STDERR] find: invalid predicate `-delete'
    
    LSWS is not configured to use the cache feature.
  18. mistwang

    mistwang LiteSpeed Staff

    It should find all files under all sub-directories.
  19. mistwang

    mistwang LiteSpeed Staff

    looks like your find command does not support "-delete" action?
  20. ffeingol

    ffeingol Member

    Sorry, but this is still doing absolutely nothing. I've changed the:

    Code:
    # /bin/sh
    
    to

    Code:
    # /bin/sh -x
    
    So the execution gets logged to the stderr log. From the logs it's in the right directory and looping through all the various sub-directories but nothing actually gets deleted. If I manually run the find command as a -print instead of -delete it does print the files that should get deleted.

    Very stumped.

Share This Page