Home Commentary Tech Check: Building from the Ground Up 9.11.14

Tech Check: Building from the Ground Up 9.11.14


by Jeremy Lesniak

often recall the scenario of a client from years gone by. We repurposed some old hardware in multiple locations, yielding not one, not two, but nine copies of the client’s mission-critical data. I apply the same strategy with many clients today, never relying on a single physical location for backup. I can’t tell you how many Irene horror stories I‘ve witnessed. To tell people who just lost their home that they’ve also lost their business data and family photos? Let’s just say that I cried with them.

Unfortunately, a business we work with suffered a catastrophic failure recently on its main server. We’d seen some anomalies, so that some measures had been put in place, but they didn’t suffice. The business had five layers of redundancy in its backup. How many of them, after the failure, still held good data? One.

Backup is great, even critical, but it only backs up the folders or files you tell it to. Files move. They become corrupt or even infected. Nightly, secure backups of corrupt or infected files don’t help anyone. That’s what we had in four of this business’  five locations.

I’ll bet you think this is turning into another cautionary tale about backup, huh? You’re wrong! You see, there are actually positive aspects to such a meltdown, though they are certainly dwarfed by the negatives. As we work with this client to rebuild everything, we’re able to make more intelligent choices about the way things are set up. From the people to the data, to how it moves and how it’s saved, we can take the opportunity to make the best choice for the way things will be now, rather than how things were when they were set up.

I’m not proposing that you set fire to your servers. I’m encouraging you take a Cartesian, rational approach to your technology from time to time. Take a step back, take stock of what you have, and see if it makes sense. Could it be improved? The answer is probably yes.

One of my favorite times at our office is when we hire a new employee. I feel we do a great job at Vermont Computing with building and maintaining processes that work well for us and our clients. The problem is that it can be really hard to consider other perspectives or alternatives, especially when you’ve been swimming in your own processes for 13 years. That makes it tough to get better.

I encourage these new employees to come up with ideas for improvement as they learn how we do things currently. It’s really the best time for that encouragement, as their ideas often lead to questions that help them learn how things operate. It can be just as valuable for existing staff, as answering the new hire’s questions can reinforce a rationale or encourage re-examination.

These discussions and evaluations don’t have to be limited to technology. And they’re certainly not just for businesses. While there can be benefit in hiring an industry consultant to examine things, some of the best ideas come from those outside the industry. In fact, at Vermont Computing, some of our best technical processes came from our least technical staff.

Performing an audit on your technology just makes sense. It’s the same logic that many people use when getting checkups on their insurance, investments, cars and teeth. My favorite saying, “You don’t know what you don’t know,” applies well here. Play out scenarios: what would you do if your server went down, your car was stolen, or your teeth fell out?

As terrible as it’s all been, the business referred to earlier will look back on its travails and see their positive aspects: the ways the company moved forward, the ways the reduction in problems and bottlenecks will contribute to a better organization in the future. Don’t wait for catastrophe to teach you its lessons, though. At home or work, pull together a few people and pose the scenarios that you are most scared to consider. Data breach? Identity theft? Fire? Consider the worst scenarios and then plan for them. My career has been full of people wishing they’d done just that, proving this to be the best piece of advice I could ever write: Please don’t wait.