Differences

This shows you the differences between two versions of the page.

--- litespeed_wiki:cache:litemage:troubleshooting:warmup [2017/05/04 14:39]
Jackson Zhang [Warm Up Still Not Working]
+++ litespeed_wiki:cache:litemage:troubleshooting:warmup [2019/06/25 19:54] (current)
Jackson Zhang [Magento System Scheduler ''litemage_warmup_cache'' Hangs]
@@ Line 52: / Line 52: @@
 <code>
 * * * * * /bin/sh /home/magentouser/public_html/cron.sh cron.php
+</code>
+To verify if the cron is running ok, you can check cron log:
+<code>
+vi /var/log/cron
+</code>
+And you should see something like:
+<code>
+Oct 25 18:59:01 test CROND[12606]: (magentouser) CMD (/bin/sh /home/magentouser/public_html/cron.sh cron.php)
 </code>
@@ Line 64: / Line 74: @@
 ./mage--5/mage---internal-metadatas---160_LITEMAGE_WARMUP_META
 </code>
+**Note1:** var/cache directory is used only if user is using file system based cache. if you use redis, you don't see data there.
 Warmup files showing up means everything is okay while not appearing indicates that warmup is not being executed. If this is the case, something is wrong with your LiteMage warmup configuration.
@@ Line 217: / Line 229: @@
 ===== Warm Up Still Not Working =====
-If you have tried all of the above and LiteMage is still not working properly and showing the Warm-up List details, it is likely that incorrect values in Magento's Cron (Scheduled Tasks) setting is blocking cron jobs.
+==== Magento System Cron issue ====
+If you have triell of the above and LiteMage is still not working properly and showing the Warm-up List details, it is likely that incorrect values in Magento's Cron (Scheduled Tasks) setting is blocking cron jobs.
 We suggest returning these settings, located under **System > Configuration > Advanced > System > Cron (Scheduled Tasks)**, to their defaults below to fix the problem.
@@ Line 232: / Line 245: @@
 {{ :litespeed_wiki:cache:litemage:troubleshooting:litemage-magento-cron-schedule-default.png?nolink |}}
+You can also enable debug log, purge the cache and check the log to see if the crawler is running, expecially during the crawler scheduled time, such as 2-4, 12-14, 22-24, 32-34, 42-44, 52-54.
+ tail -f var/log/system.log
+You should be able to see log for cache purge:
+-10-27T20:31:04+00:00 DEBUG (7): LiteMage (4) Reinit Delta queue [time=0.85439600 1509136264] tags=
+-10-27T20:31:04+00:00 DEBUG (7): LiteMage (3) SetPurgeHeader: * (triggered by purgeAllByAdmin)
+then crawler should start running like the following and run for a couple of minutes until all done.
+-10-27T20:34:15+00:00 DEBUG (7): LiteMage [litemage_runner:103.16.17.40:52630:76984300] (2) ****** PRECHECK route_action [catalog_category_view] /health.html
+-10-27T20:34:15+00:00 DEBUG (7): LiteMage [litemage_runner:103.16.17.40:52626:80860200] (2) ****** PRECHECK route_action [catalog_category_view] /special-deals.html
+-10-27T20:34:15+00:00 DEBUG (7): LiteMage [litemage_runner:103.16.17.40:52628:82403400] (2) ****** PRECHECK route_action [catalog_category_view] /fitness.html
+-10-27T20:34:15+00:00 DEBUG (7): LiteMage [litemage_runner:103.16.17.40:52624:88625900] (2) ****** PRECHECK route_action [cms_index_index] /
+-10-27T20:34:16+00:00 DEBUG (7): LiteMage [litemage_runner:103.16.17.40:52632:07812400] (2) ****** PRECHECK route_action [catalog_category_view] /beauty.html
+-10-27T20:34:25+00:00 DEBUG (7): LiteMage [litemage_runner:103.16.17.40:52624:88625900] (3) Set Header X-Litespeed-Tag: G.9,S.1
+-10-27T20:34:25+00:00 DEBUG (7): LiteMage [litemage_runner:103.16.17.40:52624:88625900] (3) Set Header X-Litespeed-Cache-Control: public,max-age=43200,es
+...
+...
+...
+If nothing shows in the log, then there is still something wrong either on configuration or system cron job.
+You should also check cron process
+  ps -ef | grep cron.php
+You should be able to see something similar like the following, if not, most likely your system cron has some problem.
+  dealwel+ 4489 1 10 04:23 pts/0 00:00:17 /usr/local/lsws/fcgi-bin/lsphp5 /home/dealwellne/domains/example.com/public_html/cron.php
+==== cron.sh script issue ====
+We did see some case that system cron seems to be showing in the cron log in /var/log/cron but actually there is no such progress running. It did start to run when we change to documentroot and manually run the cron job.
+vi /home/dealwellne/domains/example.com/public_html/
+   cd /home/dealwellne/domains/example.com/public_html/
+   sudo -u dealwellne ./cron.sh
+then we modified the cron.sh script to reflect the above and everything runs ok.
+  vi cron.sh
+  #!/bin/sh
+  # location of the php binary
+  if [ ! "$1" = "" ] ; then
+  CRONSCRIPT=$1
+  else
+  CRONSCRIPT=cron.php
+  fi
+  #PHP_BIN=`which php`
+  PHP_BIN=/usr/local/lsws/fcgi-bin/lsphp5
+  # absolute path to magento installation
+  INSTALLDIR=`/bin/echo $0 | sed 's/cron\.sh//g'`
+  #       prepend the intallation path if not given an absolute path
+  if [ "$INSTALLDIR" != "" -a "`expr index $CRONSCRIPT /`" != "1" ];then
+    if ! ps auxwww | grep "$INSTALLDIR""$CRONSCRIPT" | grep -v grep 1>/dev/null 2>/dev/null ; then
+        echo "hello world"
+        echo "$PHP_BIN $INSTALLDIR$CRONSCRIPT"
+        cd /home/dealwellne/domains/example.com/public_html/
+        $PHP_BIN ./cron.php &
+  #       $PHP_BIN "$INSTALLDIR""$CRONSCRIPT" &
+    fi
+  else
+    if  ! ps auxwww | grep " $CRONSCRIPT" | grep -v grep | grep -v cron.sh 1>/dev/null 2>/dev/null ; then
+        $PHP_BIN $CRONSCRIPT &
+    fi
+  fi
+==== Magento AOE Scheduler doesn't mark completion of ''litemage_warmup_cache''====
+[[https://github.com/AOEpeople/Aoe_Scheduler|Magento AOE Scheduler extension]] sits on top of Magento's default cron functionality allowing the users to manage the jobs and visualize the timeline.
+LiteMage works with AOE scheduler most of the time. However, occasionally, Magento AOE scheduler sometimes believers the LiteMage warmup process is always running while it has stopped by the max execution time hence AOE scheduler doesn't start next ''litemage_warmup_cache'' at all.
+If you navigate to **Magento Admin > Scheduler > Scheduler Timeline**, you might see ''litemage_warmup_cache'' highlighted either in red or orange. It means the cron is hanging and is not running propery.
+{{ :litespeed_wiki:cache:litemage:troubleshooting:781630-6-scheduler-stunck.png?800 |}}
+{{ :litespeed_wiki:cache:litemage:troubleshooting:781630-6-scheduler-stunck-2.png?800 |}}
+Navigate to **Magento Admin > System > Scheduler > Job Configuration**, select ''litemage_warmup_cache'' then in the right top corner Actions, select ''Run now'', then press the **Submit** button.
+{{ :litespeed_wiki:cache:litemage:troubleshooting:781630-8-scheduler-runnow.png?800 |}}
+Check the LiteMage cron log, like so:
+  tail -f /home/jsmith/public_html/var/log/litemage_crawler.log
+-05-29T10:14:21+00:00 DEBUG (7): LiteMage [cron:jsmith:1559124843]  Generate url map for store1 url count=5320
+-05-29T10:14:21+00:00 DEBUG (7): LiteMage [cron:jsmith:1559124843]  litemage_runner crawling  store1 urls (cur_pos:0) with cookie
+  litemage_cron=store1; Array
+  LiteMage [cron:jsmith:1559124843]   (
+  LiteMage [cron:jsmith:1559124843]       [0] => https://www.example.com/
+  LiteMage [cron:jsmith:1559124843]       [1] => https://www.example.com/business-office-writing/
+  LiteMage [cron:jsmith:1559124843]       [2] => https://www.example.com/business-office-writing/conference-business/business-card-holders/
+  LiteMage [cron:jsmith:1559124843]       [3] => https://www.example.com/business-office-writing/paper/promotional-calendars-diaries/
+  LiteMage [cron:jsmith:1559124843]       [4] => https://www.example.com/business-office-writing/desk/promotional-calculators/
+  LiteMage [cron:jsmith:1559124843]       [5] => https://www.example.com/business-office-writing/paper/promotional-notebooks/
+  LiteMage [cron:jsmith:1559124843]   )
+  LiteMage [cron:jsmith:1559124843]
+-05-29T10:14:32+00:00 DEBUG (7): LiteMage [cron:jsmith:1559124843]  litemage_runner crawling  store1 urls (cur_pos:6) with cookie
+  litemage_cron=store1; Array
+  LiteMage [cron:jsmith:1559124843]   (
+  LiteMage [cron:jsmith:1559124843]       [0] => https://www.example.com/bags-and-travel/
+  LiteMage [cron:jsmith:1559124843]       [1] => https://www.example.com/clothing-accessories/
+  LiteMage [cron:jsmith:1559124843]       [2] => https://www.example.com/usb-technology/
+  LiteMage [cron:jsmith:1559124843]       [3] => https://www.example.com/food-and-drink/
+  LiteMage [cron:jsmith:1559124843]       [4] => https://www.example.com/health-personal-care/
+  LiteMage [cron:jsmith:1559124843]       [5] => https://www.example.com/bags-and-travel/travel/backpacks/
+  LiteMage [cron:jsmith:1559124843]   )
+These results mean that the warmup did begin to run, so we know it works when started manually. A ''Run now'' action won't fix the Magento system scheduler problem, however, and the initiated cron will be stopped due to exceeding the defined **Maximum Run Time** in the configuration.
+{{ :litespeed_wiki:cache:litemage:troubleshooting:781630-9-litespeed-warmup-stats-stopped-exceeding-defined-max-run-time.png?800 |}}
+Take a look at ''litemage_crawler.log'' and you will see something like the following:
+-06-19T16:17:05+00:00 DEBUG (7): LiteMage [cron:brandin1:1560960721]  Stopped due to exceeding defined Maximum Run Time. cron meta end = Array
+This is normal, since the cron was set to stop at 5 minutes before running again at the next scheduled time. If it had stopped there completely without further running, then there would be some Magento system scheduler issue which would need to be fixed.
+By taking a look at your crontab (assuming your userid is ''jsmith'') we see that you have too many of the same crons set up which may conflict with each other:
+  crontab -u jsmith -e
+  MAILTO=""
+  SHELL="/usr/local/cpanel/bin/jailshell"
+  * * * * * /usr/local/bin/php /home/jsmith/public_html/cron.php
+  SHELL="/usr/local/cpanel/bin/jailshell"
+  * * * * * /usr/local/bin/php /home/jsmith/public_html/cron.sh
+  SHELL="/usr/local/cpanel/bin/jailshell"
+  * * * * * /bin/bash /home/jsmith/public_html/scheduler_cron.sh --mode always
+  SHELL="/usr/local/cpanel/bin/jailshell"
+  * * * * * /bin/bash /home/jsmith/public_html/scheduler_cron.sh --mode default
+  SHELL="/usr/local/cpanel/bin/jailshell"
+  * * * * * /bin/bash /home/jsmith/public_html/cron.sh
+  SHELL="/usr/local/cpanel/bin/jailshell"
+  * * * * * /bin/sh /home/jsmith/public_html/cron.sh
+  * * * * * /bin/sh /home/jsmith/public_html/cron.sh cron.php
+**NOTE**: You may need ''SHELL="/usr/local/cpanel/bin/jailshell"'' on cPanel for jailshell.
+The latest 1.9.x.x version of Magento may prefer to run ''/home/jsmith/public_html/scheduler_cron.sh'' instead of ''/home/jsmith/public_html/cron.sh''. You can find this in **Magento Admin > System > Scheduler > Insctuctions**:
+  Run scheduler_cron.sh instead cron.sh (recommended)
+  Run every minute (recommended)
+  Edit your crontab:
+  sudo crontab -u jsmith -e
+  Add following configuration:
+  * * * * * /bin/bash /home/jsmith/public_html/scheduler_cron.sh --mode always
+  * * * * * /bin/bash /home/jsmith/public_html/scheduler_cron.sh --mode default
+{{ :litespeed_wiki:cache:litemage:troubleshooting:781630-7-scheduler-instruction2.png?600 |
+}}
+Remove all duplicated cron commands and only keep the above two configurations. {{ :litespeed_wiki:cache:litemage:troubleshooting:781630-9-scheduler-litemage_warmup_cache_running.png?800 |}}
+{{ :litespeed_wiki:cache:litemage:troubleshooting:781630-9-litespeed-warmup-stats-running.png?800 |}}
+{{ :litespeed_wiki:cache:litemage:troubleshooting:781630-10-aoe-scheduler-stop-litemage_warmup_cache-3-highlight.png?|800}}
+The above-scheduled tasks view show the AOE scheduler's problem. ''litemage_warmup_cache'' was scheduled for 17:02 and run for 5 mins. However, AOE scheduler did not mark the task as "completed" at 17:07 and treats it running all the time. Then 17:12, 17:22, 12:32 etc scheduled ''litemage_warmup_cache'' never run due to the incorrect status issue for 17:02 's job.  It may need your Magento developer or AOE developer to fix such issue to make ''litemage_warmup_cache'' run without any problem until the completion.
+===== Warm Up for logged in users =====
+Normally logged in users and non-logged in users will share the same cache copies. However, when you enable **Separate Cache Copy per Customer Group** in LiteMage configuration, LiteMage will have different cache copies.
+{{ :litespeed_wiki:cache:litemage:troubleshooting:litemage-logged-in-user-warm-up-when-separatecachcopy.png?600 |}}
+The crawler warmup guide above only shows how to warm up the cache for non-logged in users. However, if you don't enable the crawler for logged-in users, the logged-in page may run slow during the first visit when there's a cache miss.
+==== Enabling Cache for Logged-in Users ====
+You should use the **Crawl on Behalf of Logged-in Users** setting in LiteMage's **Cache Warm Up** configuration. Enter user IDs from representative groups, separating them by space or comma. There is no need to place all user IDs. Any single user ID from that user group will do, since every user in that group will share the same cache copy. If you have enabled multi-customer groups, one user id from each group will do.
+For the example from the above, with ''only distinguish between logged-in vs logged-out user'' selected, all logged-in users will be in the same group, and any logged-in user ID will do the job.
+You should be able to get a user ID from the Magento admin customer menu. You can search for any particular email to find an ID.
+{{ :litespeed_wiki:cache:litemage:troubleshooting:litemage-logged-in-user-warm-up-user-id-search.png?1000 |}}
+Once you have the IDs, you can place them in **Crawl on Behalf of Logged-in Users**.
+{{ :litespeed_wiki:cache:litemage:troubleshooting:litemage-logged-in-user-warm-up-onbehalfofloggedinuser.png?600 |}}
+**Save the configuration** will make the change effective.
 ===== LiteMage Cache Warm Up configuration Reference =====
 Please refer [[ litespeed_wiki:cache:litemage:configuration:cache_warm_up|this]] for LiteMage Cache Warm Up configuration Reference