ruby on rails - Heroku Europe: hung requests in Unicorn workers -


i have app on heroku forked try out europe region support. after initial hurdles (heroku europe region: "application error" when trying fork) seems working fine.

i ran simple load test ab (apache bench) tool , expected see improvements in requests per second. however, not so:

timings 1000 requests 10 concurrent users (ab -n 1000 -c 10 <url>)

us:

connection times (ms)               min  mean[+/-sd] median   max connect:      305  349  44.4    337     719 processing:   128  356 282.8    244    2213 waiting:      127  350 283.0    238    2213 total:        442  705 280.5    610    2521 

eu:

connection times (ms)               min  mean[+/-sd] median   max connect:      125  188  47.6    175     451 processing:    67 2595 3537.9   3309   30171 waiting:       66 2591 3538.9   3309   30170 total:        207 2783 3536.2   3472   30321 

some things jump out:

  • latency (see "connect:") lower eu us, expected. good.
  • some requests in eu taking 30s ? what?

after investigation, found:

  • heroku has default request timeout of 30s: request taking longer killed. explains max 30s.
  • but why happening? looked in logs , found requests fast, began take longer until eventually, seemed "hang" in unicorn worker. workers have timeout of 15s, after worker killed , restarted master unicorn process (shows in heroku logs h13 error). assume request not retried though, resulting in final time of 30s.

i have looked @ new relic, these curious slow requests show 99% time spent in unspecified "application code (root)" (which shows separately actual db access etc.) no possibility drill down (note: i'm on free plan). there seems nothing wrong app code (it running fine on after all).

my question: how go debugging this? there configuration change solve this? missing?

update:

i tried out suggestions in comments, , in end disabled memcachier addon entirely eu (plus changed app no caching @ anymore, test).

this did not resolve unicorn timeouts (although seem occur less (!)).

i suspect 1 of add-ons provisioned in different region. retrieve memcachier url config:get:

$ heroku config:get memcachier_url 

and paste domain http://ip-lookup.net/ see where, geographically, add-on hosted.

if it's in us, need reprovision in eu:

$ heroku addons:remove memcachier $ heroku addons:add memcachier 

and should provision in right region.

if add-on provisioned in wrong region, it's bug. if so, comment here app name , can it.


Comments

Popular posts from this blog

Perl - how to grep a block of text from a file -

delphi - How to remove all the grips on a coolbar if I have several coolbands? -

javascript - Animating array of divs; only the final element is modified -