| [2010/07/15 02:12:22] @ Log started by gepetto | ||
| [2010/07/15 02:12:22] @ Quit: nasrat: *.net *.split | ||
| [2010/07/15 02:12:24] @ Quit: beata-: *.net *.split | ||
| [2010/07/15 02:13:45] @ nasrat joined channel #puppet-dev | ||
| [2010/07/15 02:18:41] @ beata- joined channel #puppet-dev | ||
| [2010/07/15 06:49:05] @ notbrien joined channel #puppet-dev | ||
| [2010/07/15 07:03:26] @ rcrowley joined channel #puppet-dev | ||
| [2010/07/15 07:24:58] @ Volcane joined channel #puppet-dev | ||
| [2010/07/15 07:27:40] <Volcane> masterzen: so this is something totally different and i am totally not intending to get it merged or anything. hence wanting it a plugin, but also want it to be as clean as possible a conn pool would be needed | ||
| [2010/07/15 07:28:36] <masterzen> Volcane: sure. Still we need a better storeconfigs (or a better query tool), yours could be the base. | ||
| [2010/07/15 07:29:31] <Volcane> perhaps :) but this will require mcollective and all sorts | ||
| [2010/07/15 07:29:34] <Volcane> so probably not :) | ||
| [2010/07/15 07:29:44] <Volcane> the query syntax is just how mongo works | ||
| [2010/07/15 07:31:01] <Volcane> its really appealing though from a load perspective, since it updates the entire doc representing the host so there is non of the expensive comparison stuff | ||
| [2010/07/15 07:31:14] <masterzen> can you expand on your usage of mcollective for the query? Are you sending the query through mcollective, which in turns finds the appropriate mongodb to issue the query? | ||
| [2010/07/15 07:31:25] <Volcane> no, mcollective has a registration system | ||
| [2010/07/15 07:31:47] <Volcane> that it can send data - anything you give a plugin - over mc to wherever there is interest for the data | ||
| [2010/07/15 07:32:10] <Volcane> so in this case I'd just have a agent on every puppet master sucking in this info and dumping it into the mongodb | ||
| [2010/07/15 07:32:31] <masterzen> yes, but how are you doing the query part? | ||
| [2010/07/15 07:32:45] <Volcane> direct against mongodb | ||
| [2010/07/15 07:32:50] <masterzen> ah ok | ||
| [2010/07/15 07:32:59] <Volcane> http://www.pastie.org/1045679 | ||
| [2010/07/15 07:33:17] <Volcane> thats my hosts being updated into mongo via mcollective | ||
| [2010/07/15 07:33:46] <masterzen> then we could abstract the query system, and use a DSL syntax for it. so we could plug-in any kind of backends | ||
| [2010/07/15 07:33:49] <Volcane> now that i installed the C based ruby extension inserts of all facts, classes, agent lists etc happens in 0.001 second | ||
| [2010/07/15 07:34:27] <masterzen> nice | ||
| [2010/07/15 07:34:41] <Volcane> and I'll just use mcollective to distribute the data to every one of my master | ||
| [2010/07/15 07:34:46] * masterzen really need to install mcollective in production | ||
| [2010/07/15 07:34:50] <Volcane> no need for replication or single write system anywhere etc | ||
| [2010/07/15 07:35:11] <masterzen> so each master has his own mongodb | ||
| [2010/07/15 07:35:24] <Volcane> yeah, well thats just my need i guess | ||
| [2010/07/15 07:35:32] <Volcane> countries can be islands, often are | ||
| [2010/07/15 07:35:43] <Volcane> so each country has its own nagios, master, activemq | ||
| [2010/07/15 07:35:48] <Volcane> puppet scheduler etc | ||
| [2010/07/15 07:36:01] <masterzen> and all the activemq are chained | ||
| [2010/07/15 07:36:04] <Volcane> yes | ||
| [2010/07/15 07:36:05] <masterzen> or linked | ||
| [2010/07/15 07:36:22] <Volcane> meshed | ||
| [2010/07/15 07:36:27] <Volcane> so if uk<->de is down | ||
| [2010/07/15 07:36:37] <Volcane> it will still work via de<->us and uk<->us | ||
| [2010/07/15 07:36:52] <masterzen> is it efficient? I mean if you send a message, it will appear to every clients connected on the topic/queue | ||
| [2010/07/15 07:37:12] <Volcane> it only appear to clients that are interested | ||
| [2010/07/15 07:37:24] <Volcane> so say i run the client, only the client gets responses not everyone | ||
| [2010/07/15 07:37:30] <Volcane> activemq does that routing | ||
| [2010/07/15 07:38:13] <masterzen> so if no clients are subscribed to the topic on one amq, this amq doesn't get the message? | ||
| [2010/07/15 07:38:21] <Volcane> yup | ||
| [2010/07/15 07:38:26] <masterzen> that's cool | ||
| [2010/07/15 07:38:58] <Volcane> if i enable registration, but not the listening part | ||
| [2010/07/15 07:39:11] <Volcane> only traffic is node -> activemq -> bitbucket | ||
| [2010/07/15 07:39:20] <Volcane> and it becomes a keep alive for firewalls more or less :P | ||
| [2010/07/15 07:43:57] <Volcane> sooo, how to make a connection pool? | ||
| [2010/07/15 08:55:56] @ londo__ joined channel #puppet-dev | ||
| [2010/07/15 08:56:42] @ lak joined channel #puppet-dev | ||
| [2010/07/15 09:00:52] @ Quit: londo: Ping timeout: 276 seconds | ||
| [2010/07/15 09:54:33] @ plathrop-away is now known as plathrop | ||
| [2010/07/15 09:57:40] @ ReinH joined channel #puppet-dev | ||
| [2010/07/15 10:25:56] <masterzen> !seen markus | ||
| [2010/07/15 10:25:56] <gepetto> masterzen: markus was last seen 1 day, 14 hours, 37 minutes and 19 seconds ago, quitting IRC (Ping timeout: 245 seconds) | ||
| [2010/07/15 11:14:32] <plathrop> !seen jamesturnbull | ||
| [2010/07/15 11:14:32] <gepetto> plathrop: jamesturnbull was last seen 19 hours, 32 minutes and 50 seconds ago, saying "jhelwig: never watched it - I had to go there when I worked at a casino years ago" in #puppet | ||
| [2010/07/15 11:20:42] <Volcane> well searching works fine against mongodb as described. just need a way to make a db pool | ||
| [2010/07/15 11:29:39] <Volcane> simply using @@ vars for the handles in the parser function works, but thats fugly | ||
| [2010/07/15 12:03:40] @ Quit: lak: Quit: lak | ||
| [2010/07/15 12:03:51] <duritong> masterzen: ping | ||
| [2010/07/15 12:04:01] <masterzen> duritong: pong :) | ||
| [2010/07/15 12:04:07] <duritong> :) | ||
| [2010/07/15 12:04:20] <duritong> I'm trying to debug #4247 | ||
| [2010/07/15 12:04:20] <gepetto> duritong: #4247 is http://projects.puppetlabs.com/issues/show/4247 "Puppet - Bug #4247: puppet master 2.6 is leaking memory and getting stalled - Puppet Labs" | ||
| [2010/07/15 12:04:35] * masterzen reading the bug entry | ||
| [2010/07/15 12:04:57] <duritong> and I thought that puppet-test might be maybe something to kick off catalog compiles? | ||
| [2010/07/15 12:05:42] <masterzen> ouch | ||
| [2010/07/15 12:06:04] <masterzen> duritong: yes, in ext/ there is puppet-test a binary that can trigger catalog compilation | ||
| [2010/07/15 12:06:20] <masterzen> unfortunately the shipped puppet-test doesn't work with REST, you need mine | ||
| [2010/07/15 12:06:25] <duritong> the weird issue is that I don't have any output what the master is doing | ||
| [2010/07/15 12:06:40] <masterzen> it means it is certainly looping like mad | ||
| [2010/07/15 12:06:55] <masterzen> I suggest you strace it to see what it does | ||
| [2010/07/15 12:06:57] <duritong> I try to attach with strace and probably I can figure it out that way | ||
| [2010/07/15 12:06:59] <duritong> yep | ||
| [2010/07/15 12:07:21] <masterzen> then gdb it (with the ruby introspection stuff) to see where it is spending mem and time | ||
| [2010/07/15 12:07:44] <duritong> yeah I think I have gdb still installed with the ruby stuff | ||
| [2010/07/15 12:10:52] <duritong> wow | ||
| [2010/07/15 12:11:09] <duritong> it loops on stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1892, ...}) = 0 | ||
| [2010/07/15 12:11:10] <masterzen> you can also check this ltrace version: http://github.com/ice799/ltrace/tree/libdl | ||
| [2010/07/15 12:11:35] <masterzen> duritong: strange! | ||
| [2010/07/15 12:11:40] <duritong> and that's a link to /usr/share/zoneinfo/Europe/Zurich | ||
| [2010/07/15 12:11:46] <duritong> as usual on centos boxes | ||
| [2010/07/15 12:11:50] <duritong> letme copy the content | ||
| [2010/07/15 12:12:27] <masterzen> the process might be doing some Time.now | ||
| [2010/07/15 12:12:34] <masterzen> it needs to get the timezone info | ||
| [2010/07/15 12:12:34] <duritong> yep | ||
| [2010/07/15 12:12:37] <masterzen> brb | ||
| [2010/07/15 12:12:48] <duritong> but that works fine in irb | ||
| [2010/07/15 12:14:13] <duritong> the same with a copy of the file | ||
| [2010/07/15 12:17:37] * duritong updated bug report | ||
| [2010/07/15 12:17:51] * duritong tries to get the ruby stacktrace with gdb | ||
| [2010/07/15 12:23:40] <masterzen> duritong: you can also try to "instrument" puppet manually (put some "puts" around do_find and indirector/compiler.rb) maybe you'll be able to find where it is going | ||
| [2010/07/15 12:25:32] <masterzen> duritong: I'm going offline for a couple of hours, see you | ||
| [2010/07/15 12:26:36] <duritong> I use now: http://weblog.jamisbuck.org/2006/9/25/gdb-wrapper-for-ruby | ||
| [2010/07/15 12:26:48] <duritong> masterzen: yeah I'll do | ||
| [2010/07/15 13:03:18] @ DarkSinclair joined channel #puppet-dev | ||
| [2010/07/15 13:14:55] * duritong found the cause, but no idea so far of how to fix it | ||
| [2010/07/15 13:15:32] * Volcane got his pool working | ||
| [2010/07/15 13:15:46] <duritong> Volcane: :) | ||
| [2010/07/15 13:15:53] <Volcane> <% nodes = Puppet::Util::MongoQuery.instance.find_nodes({"facts.customer" => "rip"}) %> | ||
| [2010/07/15 13:16:00] <Volcane> nice array of all my machines | ||
| [2010/07/15 13:16:29] <Volcane> with access to facts, classes, etc | ||
| [2010/07/15 13:16:32] <duritong> Volcane: but isn't Mongo not fully persistent? | ||
| [2010/07/15 13:16:52] <Volcane> this data is more or less transient | ||
| [2010/07/15 13:17:01] <Volcane> if i loose it it takes mcollective a minute to recreate it all | ||
| [2010/07/15 13:17:39] <duritong> ok | ||
| [2010/07/15 13:20:46] <Volcane> my only gripe is that Puppet::Util::* stuff cant be copied out and expect to just work with require via plugins in modules | ||
| [2010/07/15 13:20:53] <Volcane> seems that dir isnt actually in the load path | ||
| [2010/07/15 13:21:11] <Volcane> some magical requires happen with parser functions that doesnt happen with Util::* | ||
| [2010/07/15 13:22:48] @ lak joined channel #puppet-dev | ||
| [2010/07/15 13:23:36] <Volcane> so i guess i might file a feature request for that | ||
| [2010/07/15 13:23:45] <duritong> yeah | ||
| [2010/07/15 13:23:57] <duritong> ok something is broken with passing around exported resources in 2.6 | ||
| [2010/07/15 13:24:07] <duritong> but I dunno yet where... | ||
| [2010/07/15 14:46:06] <duritong> masterzen: thanks for the tip | ||
| [2010/07/15 14:46:15] <duritong> masterzen: I figured the problem out | ||
| [2010/07/15 14:46:28] <duritong> masterzen: actually there are two problems | ||
| [2010/07/15 14:46:38] <masterzen> duritong: what are they? | ||
| [2010/07/15 14:46:51] <duritong> masterzen: for the main one I have the solutions, but not (yet) for the second one | ||
| [2010/07/15 14:46:56] <duritong> masterzen: I updated the bug report | ||
| [2010/07/15 14:47:02] <masterzen> duritong: ok I'll read it | ||
| [2010/07/15 14:48:30] <masterzen> duritong: ok, I understand | ||
| [2010/07/15 14:48:57] <duritong> masterzen: I'm going out for food now. I will probably not try to figure out the second problem (infinite loop in inspection), as compilation now continues and I have now another problem to fix in my manifests | ||
| [2010/07/15 14:48:58] <masterzen> duritong: what is strange is that I tried with storeconfigs on (but with a blank db) and it worked. | ||
| [2010/07/15 14:49:11] <duritong> masterzen: yeah I remembered that | ||
| [2010/07/15 14:49:16] <masterzen> duritong: ok, see you later | ||
| [2010/07/15 14:49:18] <duritong> and that's indeed strange | ||
| [2010/07/15 14:49:27] <duritong> masterzen: but did you use exported resources? | ||
| [2010/07/15 14:49:31] <masterzen> yes | ||
| [2010/07/15 14:49:32] <duritong> because it fails only there | ||
| [2010/07/15 14:49:41] <duritong> strange | ||
| [2010/07/15 14:49:49] <masterzen> I'll do more test tomorrow | ||
| [2010/07/15 14:49:55] <duritong> ok :) | ||
| [2010/07/15 15:02:01] <jamesturnbull> plathrop: here now | ||
| [2010/07/15 15:02:02] <jamesturnbull> masterzen: markus is in Austin, Texas | ||
| [2010/07/15 15:02:25] <jamesturnbull> masterzen: so he's access is a bit ... sporadic | ||
| [2010/07/15 15:03:20] <Volcane> why? did his ADSL connection forget its papers and they're chasing it around with guns? | ||
| [2010/07/15 15:09:18] <jamesturnbull> he's at a session and I think the wireless is shite or somethijng | ||
| [2010/07/15 15:09:29] <Volcane> ah :) | ||
| [2010/07/15 15:19:46] <Volcane> nice, thats 3 masters all being fed data into mongodb and queries from manifests/templates working against that. all nicely distributed etc | ||
| [2010/07/15 15:31:20] @ Quit: notbrien: Quit: notbrien | ||
| [2010/07/15 15:59:00] @ Quit: DarkSinclair: Quit: Page closed | ||
| [2010/07/15 16:13:40] @ github joined channel #puppet-dev | ||
| [2010/07/15 16:13:40] <github> puppet: master James Turnbull * 4ce33fd (3 files in 1 dirs): Fixed #4249 - Updated SUSE packaging specifications - http://bit.ly/c6CCgA | ||
| [2010/07/15 16:13:40] <gepetto> github: puppet: #4249 is http://projects.puppetlabs.com/issues/show/4249 "Puppet - Bug #4249: suse init / spec files were outdated, and did not work on new versions of sles - Puppet Labs" | ||
| [2010/07/15 16:13:40] @ github left channel #puppet-dev () | ||
| [2010/07/15 16:29:13] @ Quit: lak: Quit: lak | ||
| [2010/07/15 16:51:45] @ Quit: londo__: Ping timeout: 240 seconds | ||
| [2010/07/15 16:51:53] @ londo__ joined channel #puppet-dev | ||
| [2010/07/15 17:07:32] <duritong> is jesse in irc? | ||
| [2010/07/15 17:10:33] @ jmccune joined channel #puppet-dev | ||
| [2010/07/15 17:11:38] <plathrop> jamesturnbull: I was going to bother you about Nagios. I hear you are a bit of an expert...? | ||
| [2010/07/15 17:12:01] <jamesturnbull> plathrop: might have some idea... :) | ||
| [2010/07/15 17:12:50] <plathrop> jamesturnbull: will take to PM since it is off-topic | ||
| [2010/07/15 19:00:59] @ Quit: jmccune: Ping timeout: 252 seconds | ||
| [2010/07/15 19:12:31] @ plathrop is now known as plathrop-away | ||
| [2010/07/15 19:36:29] @ Quit: rcrowley: Quit: rcrowley | ||
| [2010/07/15 20:18:19] @ lak joined channel #puppet-dev | ||
| [2010/07/15 21:34:27] @ jmccune joined channel #puppet-dev | ||
| [2010/07/15 21:50:34] @ github joined channel #puppet-dev | ||
| [2010/07/15 21:50:34] <github> puppet: tickets/master/yumrepocost James Turnbull * 991ba00 (1 files in 1 dirs): Added cost parameter to the yumrepo type - http://bit.ly/dpm2NF | ||
| [2010/07/15 21:50:34] @ github left channel #puppet-dev () | ||
| [2010/07/15 21:50:55] @ github joined channel #puppet-dev | ||
| [2010/07/15 21:50:55] <github> puppet: tickets/master/deprecation James Turnbull * 5cc5313 (1 files in 1 dirs): Fixed yumrepo type deprecation wanring ... - http://bit.ly/bTmOCq | ||
| [2010/07/15 21:50:55] @ github left channel #puppet-dev () | ||
| [2010/07/15 22:35:40] @ Quit: jmccune: Ping timeout: 276 seconds | ||
| [2010/07/15 23:21:22] @ Quit: lak: Quit: lak | ||
| [2010/07/15 23:27:20] @ jmccune joined channel #puppet-dev | ||
| [2010/07/15 23:31:31] @ Quit: jmccune: Ping timeout: 240 seconds |
Generated by irclog2html.py 2.6 by Marius Gedminas - find it at mg.pov.lt!