Jeff's Professional Side

Last night Bobby, Jason, and I returned to our old habit of attending a monthly meeting of the Atlanta UNIX Users Group. Returning speaker Ryan Matteson gave a rather informative presentation on the topic of Centralized Logging with syslog-ng. I’ve used syslog-ng on and off over the years, and we recommend the GPLv2 version to our services and support customers for use in conjunction with the syslog listener built into OpenNMS. Despite all this, I had never had a proper introduction to syslog-ng, so this meeting was one not to be missed.

Taking a cue from DJ’s recent post, I’m putting my notes from the talk online. Ryan has also put his slides online, so there’s not much marginal value in the following except for a couple of verbal comments that I rolled in.

Read more…

Geeky atlanta, auug, Network Management, OpenNMS, opensource, syslog-ng

The second annual Atlanta Linux Fest took place this past Saturday, the 19th September 2009. IBM was gracious enough to provide the facility, and Canonical and eApps also provided major sponsorship. The show was a success by all measures, well exceeding the organizers’ expectations. The official attendance numbers have not been published, but I have heard numbers between 600 and 700 — not bad for a show only in its second year and that fell on a very rainy day!

At Nick’s request, I had convinced Tarus to give a talk, and I pretty much unilaterally decided that OpenNMS would also be an exhibitor. We scored an absolutely prime spot in the exhibit area and drew in lots of traffic. Our good friends Mita and Robbie at Presentation Rentals (which sponsors ANSMTUG monthly) got us a great deal on a big monitor on which we ran our loop. My amazing wife volunteered to help out with the booth, her first time ever “wearing the shirt” though she’d hung out with us last October at a show in London. We always like to have as many community members representing as we have OpenNMS Group employees, so I recruited local friends Bobby and Robert to help man the booth. I’m confident that each of them fully earned a 100% commission on the free software he sold that day.

A plethora of Papa John's pizza pies, constituting lunch at Atlanta Linux Fest 2009

This show was the project’s first in Atlanta, and as happens in every new city, people came out of the woodwork who use OpenNMS to keep the IT infrastructure running smoothly at local companies that we previously had no idea were part of our user community. It turns out that Georgia Tech, which hosted last year’s Dev-Jam, is a user, as is Georgia Power. Less widely known users that we learned about include the company that provides the data underlying Google Maps‘ and other providers’ integrated roadway traffic services. I hope that some of these people will join the Order of the Blue Polo, especially since both my wife and Bobby were modeling the shirt in question. It’s fitting, and a testament to the show organizers’ excellent taste, that lunch came from a company represented in the OBP.

It was such a busy day for us that nobody had time to snap any pictures of our booth! We were fortunate enough to have, in the booth adjacent to ours, shutterbug Jane Ullah, who captured three of us loafing during a lull in the show traffic. If some images emerge that paint us in a less idle light, I’ll update this post to include them

Food, Geeky, OpenNMS atlanta, atlantalinuxfest, atllinuxfest, canonical, family, Food, ibm, linux, Network Management, OpenNMS, opensource

I like to shoot. I’ve been lucky enough so far never to have needed to shoot an animal for food or a fellow human being out of self defense. Although I consider myself a moderate on most issues, the right to keep and bear arms is a freedom that I cherish. That’s probably due in part to the fact that I grew up in a fairly rural area, and from a fairly young age tagged along with my father to the gun range for target practice. I was probably ten years old when I first personally shot anything, and I seem to recall having reliably hit a target at that age. After a few years of no shooting at all, I pretty easily earned my Rifle Shooting Merit Badge at Boy Scout camp. That was more years ago than I like to admit, and unless you count paintball I haven’t discharged a firearm since.

That is, until last weekend, when I visited family for a wedding. (Insert hillbilly joke here.) My uncle, who is a part-time competitive marksman (and father of the bride), took dad and me to the pistol range for a few hours of turning neatly cast bits of lead and brass into irregularly flat ones.

My guest pass to Bluegrass Sportsmens League

This trip was my first time doing practical shooting, in which time counts as much as accuracy. It turns out that from the holster at thirty yards I can reliably hit four rectangular steel plates, each about the size of an average Dell notebook computer, and then a larger fifth plate, in just over six seconds. That’s about a second slower than my dad and about three seconds slower than my uncle, but not a bad time for someone who hasn’t handled a loaded weapon in about 536 million seconds. More amazing is that I can do this with no damage to that flap of skin between the thumb and forefinger of the dominant hand, a part of the less experienced shooter that Glock actions seem to take particular delight in biting.

I’m now going to have to join my wife in applying for a Georgia firearms license (Georgia, my home these days, is one of the few United States that requires a license to carry openly) and see if I can get that time under six seconds before my next visit to BGSL.

Geeky 2ndamendment, family, marksmanship

One of the lists that I read, and occasionally post to, is asterisk-dev. I usually don’t have time to follow the discussion closely, but this morning Russell posted (and blogged) about the possibility of the Asterisk project getting involved with Google’s Summer of Code this year. Russell solicited input from the list on projects that people would be interested to see worked on and, more importantly, willing to mentor. The OpenNMS project (which my employer sponsors and helps maintain) was a mentoring organization in the 2008 GSoC, so I sat up and paid attention.
Read more…

Geeky, OpenNMS, Software, VoIP Asterisk, community, diaf, digium, ecosystem, foad, OpenNMS, opensource, stewardship, troll

I wanted to to a little more than just re-tweet the link that Taylor posted earlier. I don’t have time to do a full-blown post, but here are some screen shots of the pseudo-holographic Flash gizmo that GE is doing as part of its Plug Into the Smart Grid campaign at work.
Read more…

Geeky cool, deadtree, flash, ge, generalelectric, ironic, plugintothesmartgrid, retweet, taylorbanks, twitter

I was going to hold off until Tarus had a chance to respond to Luke’s post, but I’m guessing Tarus is away from the intarwebs today. After having trouble posting my comment (probably just too long to be accepted) on Luke’s blog, I’m doing it as a post here on my own.

Without presuming to speak for Tarus (Ben and I both work with Tarus and so are quite familiar with his philosophies) I’d like to lay out a few places where I think Luke’s post missed the mark. These thoughts are ones that I really wanted to get across in person during our brief conversation in January after Luke’s talk at CloudCampATL, but I didn’t want to hog his time.

My points follow.

1. Open-core is not invalid as a business model, it’s just not the same as open-source

Tarus’ campaign is not against the open-core business model itself. It’s against open-core companies that want to have it both ways by playing the necessary-feature-withholding game but still calling themselves “open source companies”. It’s not about hate but about calling something what it is.

2. Joint ownership of copyright and why we as a project sponsor pursued it

As the maintainers of OpenNMS, we do require all contributors to execute an agreement that concerns copyright. It’s not about attribution or assignment, though, as assignment in particular is essentially impossible under current U.S. copyright law. It’s actually about declaring a shared or joint copyright. A contributor who executes the OpenNMS Contributor Agreement (which Tarus blogged about here) forfeits exactly zero rights while simultaneously granting us the same set of rights that the contributor has. The contributor is still free to do anything he wishes with his contribution — license it commercially, give it away under a non-GPL-compatible license, print it out and roll a joint with it, or (gasp) also contribute it to Hyperic. That point was key in our selecting this agreement, which as Ben pointed out earlier is literally the Sun Contributor Agreement with e-mail and fax information added and “Sun Microsystems”, “OpenSolaris”, and “California” changed to “The OpenNMS Group”, “OpenNMS”, and “North Carolina” respectively. We were incredibly hesitant to add this kind of requirement, and we didn’t do so unilaterally. When we went shopping for an agreement, we created a committee of non-employees from the Order of the Green Polo (the secret society of core OpenNMS contributors) to do the research and waited months for them to make a recommendation — they all have day jobs, after all. We knew that we ran the risk of pissing off and maybe even running off valuable community members, something that thankfully did not happen. Even with OCAs on file for all our contributors, we still don’t hold all the copyright since we inherited the code from the original creator, Oculan (formerly PlatformWorks, then Atipa, now part of Raritan through subsequent acquisitions). Raritan doesn’t want to sell us the intellectual property for one dollar, so we couldn’t do anything evil with the code today even if we had the desire to do so.

At the end of the day, our motivation for adding the OCA was about protecting the community, not about enriching ourselves. Having all the rights conferred under U.S. copyright law is the only way that we as a company can take legal action to protect the community from having its work (allegedly!) ripped off by people whose primary interest actually is self-enrichment. it’s incredibly tricky to pursue a GPL-violation lawsuit when you don’t hold copyright on your entire code base. Having an OCA for each contributor means that if the project ever has to appear in court, we don’t have to go broke flying everybody who’s ever contributed code to the venue. One other thing that a folder full of OCAs allows us to consider doing is offering patent indemnification to our support customers.

3. Making money selling software, and pragmatism

As far as development costing money, The OpenNMS Group is in exactly the same boat as Reductive Labs. We’re not running a charity either, and we’ve all got heads full of ideas that we just don’t have the resources to translate into code. It takes real discipline when you love the project as much as I do to put off work on an amazingly cool feature until we find somebody willing to fund it, and go work on weatherizing the damn house instead (which I should be doing right now). Would we ever sell some software? We’ve talked about it, but we would never sell OpenNMS to end users. If we held all the copyright would we maybe license the code commercially to, hypothetically, HP as the foundation of a completely redone OpenView Network Node Manager? Only if the community were on board, and the version HP got would be exactly the same as the GPL version, and we’d plow almost every penny of profit back into the project the same as we do today. This all in keeping with our promise that “OpenNMS will never suck, and it will always be free.”

4. Supported binaries and what we’re willing to support

Luke refers to a model of producing supported binaries — which we do, but we also are just as happy to support a customer who builds everything from source. There’s no shrink-wrapped version of OpenNMS is what I’m getting at here, and if there ever is, it won’t affect our willingness to support people who roll their own. We even support customers who make substantial changes to the code provided they purchase a Developer Support agreement in addition to Basic or Enterprise support. Digium is going through a transition to a similar model right now — they’re about to start offering support contracts for GPL Asterisk where they previously offered support only as part of the cost of the shrink-wrapped Asterisk Business Edition. While I wish we had something like Digium’s hardware revenue stream to draw from, I do think that their move is something of a validation of our model.

5. The greater sin is (C), both of the above

Last item, and this one’s actually way more than just the semantic distinction that I thought it would be when I wrote this paragraph the first time. Luke asks, “Is it a greater sin to only accept patches to your product if the contributor is willing to assign copyright to your commercial company, or to produce some closed-source code?” I would like to point out that the open-core guys, in our sector at least, are doing both. See Zenoss‘ and Hyperic’s terms. In so doing, these guys are accepting code from the community and reserving the right to roll it into their closed-source code.

Geeky, OpenNMS, Software copyright, gpl, hyperic, opencore, OpenNMS, opensource, puppet, reductive, zenoss

In an effort to be helpful, meet some new folks, and figure out which aspects of the current cloud computing craze will actually be relevant in the future, I volunteered at the first CloudCampATL. Originally I was supposed to be the network administrator, but there ended up being no network to administer so I handed out shirts to the other volunteers, helped out at badge pick-up, and took some video using one of the two Flip video recorders that one of the event sponsors kindly donated, and which we gave away in a drawing at the end of the evening. Sid and I each shot some, and I’ve since spent hours trying to get the video off of my Mac and onto a sharing site. Right when I was ready to give up on YouTube, Doug McClure suggested blip.tv, so that’s where it’s going. All the videos that I’m aware of are embedded after the jump. It’s also available directly from blip by searching tag ccatl. If anybody knows who the speaker is in the Joyent video, please let me know.
Read more…

Geeky ccatl

A certain kind of person will recognize the title of this post as the name of the software suite originally known as TRAKKER, and subsequently as eHealth. That kind of person will have worked in the network management sector during the late 1990s and early 2000s and been exposed to this product line from Concord Communications. That’s no mistake – I spent five formative years working at Concord deploying, customizing, and developing various products within the suite before CA acquired the company in 2005 and I got chosen for the first round of layoffs.

Despite the title, this post is not about looking back at a piece of software that helped define an important part of my past career. It’s about looking forward to the amazing ways that OpenNMS is continuing to define both my own future and the future of organizations that are staffed by people smart enough to choose it as their primary IT management platform.

I just got off the phone with a customer who works for a non-profit regional health system in metro Atlanta. Now you understand the post title! This organization has been using OpenNMS for years as an adjunct to an expensive commercial management platform whose name rhymes with “FrogLight”. Recently the IT department hired a particularly smart and good-looking bunch of people who decided to migrate the management of all internal networks, systems, and applications to OpenNMS. These folks recognized that such an effort is best accomplished with expert help, so they engaged my employer to provide professional services and technical support.

The subject at hand for the phone call just completed was to help configure OpenNMS to monitor a pair of metrics, namely the number of files and the age of the newest file in a particular directory on a Windows file server. That directory serves as an incoming queue for job processing. If the file count exceeds some maximum number, or if the newest file is older than some maximum age, then the application consuming the queue is known to be headed for trouble, so the application administrator wants to know about it. While this problem is fairly mundane, solving it without installing extra software is hard! The SNMP agent that ships with Windows is not extensible via simple scripts, so there’s no easy solution analogous to extending the Net-SNMP agent’s MIB.

We thought at first that OpenNMS’ support for the NSClient++ quasi-agent would do the trick as a one-off, but it turns out that this approach would require extensive configuration on the managed side, and the customer wanted to avoid the change management impact of installing a package that existed nowhere else in the environment. In the end, we conscripted IIS, which was already installed and running on the managed server, into service as a stand-in agent by whipping up an ASP that prints the two metrics’ values as a simple web page. We configured OpenNMS’ HTTP Collector to pull in the values as first-class node-level performance data. From there it was a simple matter of configuring two high thresholds to compare the collected values in real time against the acceptable maximums.

Apart from knowing that physicians get cranky when they don’t get serviced quickly, I have no idea what kind of jobs pass through the queue that we’re now helping our customer better manage. All that matters from our perspective as a company is that our efforts are helping a bunch of physicians work more efficiently. That fact could come home to me someday since the nearest hospital to my house is part of the customer’s system, and I want any physician who’s working on my body to be happy with the tools at his or her disposal.

This story is just the latest coup in a series that started when the customer’s IT department decided to consolidate management and monitoring onto OpenNMS. After we ironed out the last configuration details of the support ticket at hand, my contact let me know that he would be opening several additional tickets over the coming week. It’s not that he’s having problems with OpenNMS, but that his colleagues are lining up to discuss problems that they think OpenNMS could help them solve. We’ll be here to help him stay on top of that queue as well, which means that our contact will stay happy with both OpenNMS and our support services, which help him apply OpenNMS to meet his job objectives. Because all of OpenNMS is OSD-compliant free and open-source software, its functionality comes at no cost beyond the hardware and staff required to operate it, which means that our customer’s management and board of directors stay happy. Happy management tends to have no reservations about writing a check when the time comes for support renewal.

Geeky, OpenNMS, Software Network Management, OpenNMS

This post is a bit late in coming, but I’ve been working on it since my body was somewhere over southern Utah during my flight from Atlanta (home to me) to San Francisco. Having had my nose buried in Lyle Estill’s book Small Is Possible, my mind was in Chatham County, North Carolina (home of OpenNMS Group World HQ). Walking from the head through business class back to my seat I thought that one of the guys I walked past looked familiar. For a brief instant I could see his face at Chatham Marketplace. That notion was plausible because I’ve been there a couple times with Tarus, but not at all likely. I climbed back across two neighbors to my window seat in coach, looked out again at canyon country, and got out my laptop to start writing this post.

Originally, nobody from the OpenNMS project was slated to go to LWCE. We have had a booth in the .ORG pavilion in past years — up until last year, in fact:

Page 42 of the LWCE 2008 Attendee Guide features Ben (left) and Antonio in the OpenNMS booth at LWCE 2007!

Despite having been offered a booth again, we decided to pass this year as the show has become increasingly commercialized and decreasingly about Open Source software. The purpose of my trip as I understood it was to provide technical support for a demo, to run in the Open Solutions Alliance (OSA) showcase, showing how alerts from Hyperic HQ could be turned into help-desk tickets in Concursive ConcourseSuite (formerly Centric CRM). Both Hyperic and Concursive are OSA members; The OpenNMS Group is not a member, but has independently developed integrations with both HQ and ConcourseSuite. The idea was to show off a software stack that David, in a clever play on LAMP, dubbed OUCH:

• OpenNMS
• Ubuntu
• Concursive
• Hyperic

David had built an Ubuntu virtual machine and installed OpenNMS, HQ Open Source, and ConcourseSuite Community Edition before his plate filled up and he handed off the project to me. All that was left for me to do was to set up the integrations. As luck would have it, the VM became corrupted on setup day (Monday) and I had to start completely from scratch. I’d left my Ubuntu media at home, and the Internet access both on the show floor and at my hotel was slow and unreliable. I hit up the team from Canonical for a Hardy x86 server CD. Those guys have a real penchant for giving away install media, and would not let me leave without also taking DVDs of the desktop version for both x86 and x86_64 and a pile of stickers. In exchange for their help, I promised to put an Ubuntu sticker on my Macbook Pro for the demo. Chalk one up for a project that does business without ever charging for the right to use its software, and therefore has no agenda that could put it at odds with its community of volunteer contributors.

On the way out of the hall that day, I stopped by the Hyperic booth to say “hi” to Stacey, and also got to meet Jeremy Hogan, Hyperic’s new Directory of Community Management. Jeremy lives in the RTP area, right in the back yard of our World HQ. Hopefully we’ll get him to join us for lunch before he gets whisked away to the other coast!

I spent Tuesday fighting crappy Internet access, working out operational kinks in the integrations among the three applications in the OUCH stack (with lots of great help from Josh and Ananth of Concursive), watching people walk through the OSA Showcase area without watching the demos, and wandering the show floor dodging the Dice operatives who simply could not fathom that I am employed at what is literally my dream job. Outside Canonical’s mammoth booth and the .ORG ghetto (where I ran into Josh from PostgreSQL), I spotted only one software exhibitor that actually gets what Open Source means and lives it the way we do. That vendor is OpsView, a Nagios® integrator that sells only services. The guy I talked to in their booth was Adrian, who not only works with OpsView but turns out to be a co-worker of the OGP’s own Jonathan Sartin. Truly it’s a small (Linux)world. I was happy to learn that OpsView won Best System Management Tool in this year’s Product Excellence Awards at LWCE even despite all the money that another Nagios integrator (one that tries to do a hybrid Open Source / commercial play) was spending to put people on the floor wearing penguin costumes and carrying sandwich boards.

At lunch time on Wednesday, OGP member Jason (who also cooks a damn tasty pork chop) came by for a while. He and I walked around the floor together and talked to a couple of interesting hardware vendors on the NGDC side of the show. Apart from those guys and the .ORG ghetto, I’m inclined to agree with Jay’s assessment that the show at large amounted to “a lot of vendor wankery”. After more help from David my demo was finally ready to go, so I took over the podium and the big screen. Despite looping among a pretty slide show, a video loop that I had made for LUGRadio Live USA 2008, and a demo of the actual integration, only a few people ever sat down, and probably 75% of those who did had stopped just to rest their feet. At one point I abandoned the presentation entirely and sat down to talk with two gentlemen who were more interested in substance (”So what’s your demo actually supposed to show me?”) than in flash, and in ideology (”Wait, you’re not pulling an open-source bait and switch?”) than in hype. Even so, I suspect the number of people I truly reached and engaged is larger than the number drawn in during the rest of the show by the other demos from the OSA Showcase.

I came home early on the Wednesday night redeye and felt fine about missing the last day of the show on Thursday. All in all, I think the week affirmed our assessment that the U.S. LWCE is no longer a worthwhile show for us. I’ll see you in London in October!

Geeky, OpenNMS, Software, Travel

I’ve just wrapped up a trip to Austin, Texas. This was my first visit to Austin despite my parents having lived there for over a year, so Mandy and I went out on Friday afternoon and spent the weekend at their place. We had no idea how beautiful Austin is — the hill country is breathtaking in much the same way that Dallas and Houston (both fine cities!) are not.

There’s also very good eating. From Bergstrom we went to the Eastside Café for a tasty and healthy dinner. Saturday brunch at Chez Zee left us literally lying around the living room (the crème brulée French toast is delicious and formidable) for a few hours before heading to The Oasis for drinks and an unfortunately hazy view of Lake Travis. Duly libated, we proceeded to Siena for an excellent Tuscan dinner. Sunday included a new Egoscue menu work-up with my dad and lunch at the Kerbey Lane Café, which besides good food has perhaps the coolest 1960s-retro sink counter ever in its men’s bathroom.

I spent Monday and Tuesday with one of our customers, a video game development house that’s integrating OpenNMS into the system that will monitor and manage the health of the many servers that will power a new MMO game. These guys are using our software in a way that’s not quite like anything I’ve heard of before, and it’s going to be really cool. As a testament to the flexibility of OpenNMS we were able, in just two days, to come to a good understanding of what they want to accomplish and how to approach the project.

Back to food for a moment We broke for an authentically Austinite Tex-Mex lunch at Chuy’s (whose salsa rivals the hole-in-the-wall Mexiclone place near our house) and had dinner at Rudy’s for my first-ever helping of Texas brisket barbecue. On Tuesday one of the guys took me for a quick lunch at Conans, which besides excellent pepperoni and veggie supreme pizza also boasts a really unique atmosphere that I’m told is very Austin.

When I got back from lunch on Tuesday, there was a voicemail waiting in our sales mailbox from Kathleen, the former boss who unwittingly launched my career in the network management field. She wanted to talk with somebody about using OpenNMS to replace an installation of HP OpenView Network Node Manager (HPOV NNM).

Now there’s a bit of history here — when I turned in my notice eight years ago to Kathleen, it was to go to work for a vendor whose software she had paid to train me on. She wasn’t terribly happy about that situation, but I heard that the vendor gave her a couple of free training seats as penance.

If you’ve been paying attention, you’ll realize that this voicemail arrived on 01 April. I was well primed for pranks already, having been pwned hard-core by YouTube’s masterful RickRoll and having made the OpenNMS “Enterprise Edition” price calculator for one of Tarus‘ series of blog posts. I immediately IMed Johnny to see if he had put her up to calling, but he had had nothing to do with it.

If you’re reading this, Kathleen, thanks for considering OpenNMS and I hope to see you soon in one of our training classes or as the guy doing your GreenLight!

Tuesday night I met up at Koreana with some folks from one of our other Austin customers, also a video game house, who also want to use OpenNMS in their MMO game server monitoring system! If OpenNMS keeps spreading through the video game industry at this rate, the old stand-by of blaming slow game servers for a failed raid will soon be history The bluefin sashimi was great, by the way.

Geeky, OpenNMS, SNMP, Software, Travel