Tuesday, 2010-08-03

[2010/08/03 00:09:27] @ Log started by gepetto
[2010/08/03 00:09:27] @ joevano joined channel #puppet-dev
[2010/08/03 00:27:08] <masterzen> blkperl: I think (but might be wrong) that boolean values in propreties/params are emulated with the ruby symbols :true and :false
[2010/08/03 00:29:52] <blkperl> masterzen: thanks ill find out
[2010/08/03 00:40:24] <duritong> masterzen: since I updated to latest rack I get an error on recursive empty directories in combination with mongrel
[2010/08/03 00:40:44] <duritong> it doesn't matter whether I'm on 0.25 or on 2.6 did you encounter anything similar
[2010/08/03 00:40:51] <duritong> error with nginx+mongrel
[2010/08/03 00:41:56] <duritong> nginx spits a upstream sent too big header while reading response header from upstream
[2010/08/03 00:42:08] <duritong> on http://127.0.0.1:18141/production/file_metadatas/modules/munin/plugins/modules_dir?links=manage&ignore=.ignore&&recurse=true
[2010/08/03 00:42:55] <duritong> but I increased the buffer already to 128k/25k
[2010/08/03 00:42:59] <duritong> 256k
[2010/08/03 01:13:55] <masterzen> duritong: that's strange
[2010/08/03 01:14:21] @ Quit: kubicek: Quit: kubicek
[2010/08/03 01:14:31] <masterzen> duritong: and what's the correlation with rack? mongrel/nginx shouldn't use rack at all...
[2010/08/03 01:15:00] <masterzen> duritong: can you enable nginx debug log and look to the sent headers?
[2010/08/03 01:20:22] <duritong> masterzen: I can, well I thought it's due to rack, but I assumed wrongly that mongrel uses rack. But i updated also the mongrel gem. But I'll do that with the debug log
[2010/08/03 01:20:52] <masterzen> duritong: what version of mongrel do you have?
[2010/08/03 01:22:04] <duritong> 1.1.5 and 1.1.4 and 1.0.1 but I assume 1.1.5 is used
[2010/08/03 01:22:25] <masterzen> duritong: I did most of my test with 1.1.5... it should work fine...
[2010/08/03 01:22:44] <masterzen> duritong: hmmm, rereading what you wrote: it is upstream header, not client header...
[2010/08/03 01:23:12] <duritong> well mongrel is the upstream, not?
[2010/08/03 01:23:40] <duritong> and puppet connecting to nginx is client
[2010/08/03 01:23:47] <masterzen> duritong: yes, I was just wondering if you changed the correct nginx settings (ie proxy_buffer_size )
[2010/08/03 01:24:28] <masterzen> duritong: and proxy_buffers
[2010/08/03 01:24:32] <duritong> I changed proxy_buffer_size proxy_buffers proxy_busy_buffers_size
[2010/08/03 01:24:36] <duritong> proxy_buffer_size 128k;
[2010/08/03 01:24:36] <duritong> proxy_buffers 4 256k;
[2010/08/03 01:24:37] <duritong> proxy_busy_buffers_size 256k;
[2010/08/03 01:26:47] <masterzen> hmm, can you try proxy_buffers 16 32k; (ie increase the number of buffers, not their size)?
[2010/08/03 01:27:11] <duritong> ok
[2010/08/03 01:27:51] <masterzen> duritong: you also should tcpdump the traffic between mongrel and nginx (ie tcpdump -i lo -s 0 -w capture.cap tcp port 18140) and see what mongrel returns
[2010/08/03 01:31:32] <duritong> one of the interesting things that it only happens for the following file statement in the various forms: http://git.puppet.immerda.ch/?p=module-common.git;a=blob;f=manifests/defines/module_dir.pp;h=0ff50eb7fe8275b1ff8d6ac07a01fc87bcf2cee8;hb=c180d27fdb319e19af59ae27a678bcf12fb74cfc
[2010/08/03 01:42:36] <masterzen> duritong: I'll see if I can setup a test tonight.
[2010/08/03 01:56:49] <duritong> we'll see what I can get, you gave me some good points to investigate further. thanks
[2010/08/03 02:59:33] <duritong> interesting I think I get a 404 as a repsonse with realy huge answer containing a dump of a full hash
[2010/08/03 03:23:22] <duritong> yep it looks like it dumps the whole memory into then answer
[2010/08/03 03:23:25] <duritong> weird
[2010/08/03 03:23:47] <duritong> well not the whole memory, but the answer goes over several packets and is a 404
[2010/08/03 03:24:16] * duritong needs to sleep have to investigate further tomorrow
[2010/08/03 03:48:44] @ Quit: jbooth: Ping timeout: 260 seconds
[2010/08/03 03:55:47] @ jbooth joined channel #puppet-dev
[2010/08/03 04:05:13] <masterzen> duritong: ok, if you have time tomorrow to pastie me the dump, that might help me understand where it comes/why it's there. Have a good night :)
[2010/08/03 04:24:09] @ Quit: mqr: Ping timeout: 240 seconds
[2010/08/03 07:17:43] @ notbrien joined channel #puppet-dev
[2010/08/03 10:52:44] @ plathrop is now known as plathrop-away
[2010/08/03 11:49:52] <duritong> masterzen: ok, so puppetmaster is repsonding with an imense 404 reply to that request and it sends like 7 tcp packets until nginx resets the connection
[2010/08/03 11:51:39] <duritong> and the imense reply is certainly because the HTTP/1.1 404 Could not find instances in file_metadata with '{ ... is then a huge hash
[2010/08/03 11:52:53] <duritong> so I dig into the code and try to figure out what it is dumping there
[2010/08/03 11:55:36] <duritong> ok it dumps the whole indirection_request and inspects it
[2010/08/03 12:11:25] <duritong> there you go
[2010/08/03 12:11:30] * duritong writing bug report
[2010/08/03 12:27:15] @ plathrop-away is now known as plathrop
[2010/08/03 12:36:38] <duritong> #4458
[2010/08/03 12:36:38] <gepetto> duritong: #4458 is http://projects.puppetlabs.com/issues/show/4458 "Puppet - Bug #4458: 404 error message is getting to big - Puppet Labs"
[2010/08/03 14:36:29] @ Quit: londo__: Remote host closed the connection
[2010/08/03 14:56:14] @ Quit: notbrien: Quit: notbrien
[2010/08/03 15:04:56] @ londo joined channel #puppet-dev
[2010/08/03 15:05:23] @ Quit: londo: Remote host closed the connection
[2010/08/03 15:21:51] @ jmccune joined channel #puppet-dev
[2010/08/03 15:40:02] @ kubicek joined channel #puppet-dev
[2010/08/03 16:07:42] @ londo joined channel #puppet-dev
[2010/08/03 16:09:33] @ Quit: londo: Remote host closed the connection
[2010/08/03 16:14:40] @ londo joined channel #puppet-dev
[2010/08/03 16:29:10] @ Quit: jmccune: Ping timeout: 276 seconds
[2010/08/03 16:30:55] @ jmccune joined channel #puppet-dev
[2010/08/03 17:04:15] @ Quit: jmccune: Ping timeout: 252 seconds
[2010/08/03 17:13:48] @ Quit: kubicek: Quit: kubicek
[2010/08/03 18:53:14] @ beata- joined channel #puppet-dev
[2010/08/03 19:35:52] @ plathrop is now known as plathrop-away
[2010/08/03 20:32:42] @ plathrop-away is now known as plathrop
[2010/08/03 22:15:43] @ lak joined channel #puppet-dev
[2010/08/03 22:34:04] @ Quit: lak: Quit: lak

Generated by irclog2html.py 2.6 by Marius Gedminas - find it at mg.pov.lt!