Apacheでスコアボードがフルになるエラー
Apache2.4で動かしているAPIサーバーで以下のエラーログが出力されていた。
Resource temporarily unavailable: apr_thread_create: unable to create worker threadリソースが足りないと言われているので、MaxRequestWorkersやMaxConnectionsPerChildを見直したんだけど、今度は↓のようなエラーが出力されるようになった。
[Fri Jul 26 00:01:02.461732 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 00:01:54.537697 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 00:01:55.539415 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 00:01:56.540492 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 00:01:57.541443 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 00:01:58.542447 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 00:01:59.543380 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 01:09:42.811269 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 01:09:43.813029 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers
[Fri Jul 26 01:18:50.654931 2013] [mpm_event:error] [pid 15437:tid 140446315276096] AH00485: scoreboard is full, not at MaxRequestWorkers原因はKeepaliveでした。KeepAlive Onになっていて、MaxKeepAliveRequests 100だったので、MaxConnectionsPerChildのコネクションを維持できずにエラーになっていた模様です。KeepaliveをいったんOffにしたところエラーは出なくなりました。あとでMaxKeepAliveRequestsを最適化させておこう。
AJPコネクタでエラー
先ほどのエラーは出なくなったのですが、バッチが動く朝4時に下記のエラーが出力されていました。
[Fri Jul 26 04:10:33.416964 2013] [proxy_ajp:error] [pid 8305:tid 139872377366272] (104)Connection reset by peer: AH01030: ajp_ilink_receive() can't receive header
[Fri Jul 26 04:10:33.417274 2013] [proxy_ajp:error] [pid 8013:tid 139871611606784] (104)Connection reset by peer: AH01030: ajp_ilink_receive() can't receive header
[Fri Jul 26 04:10:33.417308 2013] [proxy_ajp:error] [pid 8305:tid 139872345896704] (104)Connection reset by peer: AH01030: ajp_ilink_receive() can't receive header
[Fri Jul 26 04:10:33.417388 2013] [proxy_ajp:error] [pid 8013:tid 139871569647360] (104)Connection reset by peer: AH01030: ajp_ilink_receive() can't receive header
[Fri Jul 26 04:10:33.417467 2013] [proxy_ajp:error] [pid 8013:tid 139871548667648] (104)Connection reset by peer: AH01030: ajp_ilink_receive() can't receive header
[Fri Jul 26 04:10:33.417750 2013] [proxy_ajp:error] [pid 8305:tid 139872324916992] (104)Connection reset by peer: AH01030: ajp_ilink_receive() can't receive header
[Fri Jul 26 04:10:33.417976 2013] [proxy_ajp:error] [pid 8013:tid 139871464748800] (104)Connection reset by peer: AH01030: ajp_ilink_receive() can't receive header
[Fri Jul 26 04:10:33.418098 2013] [proxy_ajp:error] [pid 8013:tid 139871863363328] (104)Connection reset by peer: AH01030: ajp_ilink_receive() can't receive headerAJPコネクタでTomcatに接続している処理が切れてしまうみたい。
<LocationMatch "^/(***|****)/(s|n|b)/[0-9]*/[0-9a-z]*\.fs$">
ProxyPass ajp://localhost:8009/
</LocationMatch>
このProxyPassをProxyPass ajp://localhost:8009/ keepalive=onにすると接続は維持されるみたい。明日試してみよう。
