How lint made me a fortune

posted: August 13, 2017

tl;dr: Let the machine find as many mistakes as possible...

No, I don’t own a chain of laundromats. I’m referring to the software tool called “lint”, not the fluffy stuff you clean out of a clothes drier. The headline is also a slight exaggeration: I didn’t make a huge fortune, and it wasn’t entirely due to lint. But lint did help.

This is the lint I am talking about

I was introduced to lint by a veteran software engineer at a startup company that i joined shortly after university. None of my Computer Science courses had mentioned it; this is another item to add to the “things they don't teach you in school” list, although perhaps they do now. Lint, for those who don’t know, is a static code analysis tool: linters examine source code and look for mistakes and non-ideal behavior that will or might lead to problems later, at compile time or at run time when users are running the software. Problems most linters will find include: using a variable before it has been initialized, which in some languages means the variable will contain whatever garbage happens to be in that variable’s location in memory; potential overflow errors where the value stored in a variable might exceed the maximum that can be stored in that type of variable; and circular references, where something points to something which points back to itself instead of a final object.

I thought linting was great: why not let the machine find your errors? As a general rule it’s best (and cheapest) to find errors as early in the software development pipeline as possible. I also don’t want other people to find errors in my code. Nobody’s perfect, and using a linter can definitely make one’s code better.

That startup went bankrupt, and I joined a more established company that was just starting to get into the business of making high-speed Internet access networking equipment. I joined the bleeding-edge development team, whereas another team was working on what the company hoped would be the next major revenue booster, a networking device for terminating T-1 circuits and interfacing with other networking gear at the customer’s site. At the time a device like this was pretty much required to do high speed wide-area data networking. The company I joined was one of a small number vendors with this type of product available. It should have been a good seller.

Yet it was not. The product had developed a reputation among the first customers for being unreliable. There were lots of complaints from the field about the product resetting itself or otherwise interrupting the flow of data, causing outages. Sometimes the product didn’t recover on its own, requiring someone to physically cycle power on it. Customers had complained so much and so loudly that the company’s salesforce was losing faith, which can be kiss of death: when no one wants to sell a product, not much of that product is going to be sold.

The software team on this product was having a hard time solving these issues, since they were intermittent and hard to reproduce. The VP Engineering called a meeting with the software team and other developers working on other projects, to pool the best technical minds in the company. Various ideas to make things better were discussed. Eventually we asked the team working on this product: has this (primarily C) codebase ever been linted? No, it had not.

The team was dispatched to run lint on their codebase. The first pass results: 600 errors and warnings. The next instruction to the team was to immediately stop working on new feature development, and to do a “reliability improvement” release that had one requirement: get the number of lint errors and warnings down to zero.

The team did so, and shipped that release both in new devices and as a field upgrade for already installed devices. Lo and behold, the number of field complaints went way down. The salespeople were especially thrilled: they no longer got yelled at by customers, and they had a product they could sell confidently. The market for high speed Internet access started booming, and this product became a leader in its market and for years was the company’s top selling product. The company grew dramatically: a factor of five increase in sales in five years. The VP Engineering was promoted to CEO, and he named me his successor. I received some stock options in the parent corporation which owned the company I worked for, and that stock did very well for years; I was eventually able to cash in some of it. Life was good. It wasn’t all due to lint, of course, but lint played a crucial role at a very important “tipping point” moment. If we hadn’t fixed that product, the company’s outcome could have been very different.

Compilers and interpreters and languages have improved over the years, and functionality for catching some of the problems that lint used to flag is now built into them. Lint has also improved, with text editor plugins that can run lint interactively while the code is being written, to provide immediate feedback. That’s the way that I use it today, even though it throws an immediate “variable assigned to but never used” warning when I first create a variable, before I type the first line of code that uses that variable. I appreciate the immediate feedback from my old friend, lint.

Related post: Internet access fit for a king

Related post: Serendipitous software, part two