Commentary: How to treat the people that built the system you work on

Put some respect on their accomplishments

Your favorite part about the system you work on should be that it exists and has created a job for you. All the other systems built around the same time are now, somehow, dead. Every job or contract to maintain (or even replace) a system means someone found a quality solution to a real problem and figured out a way to survive long enough to build a business around it.

So start there. This might be a bit easier to appreciate if you have ever tried to build a business.

The Power of Ignorance

Feeling like the system you are working on is low quality feels great. Sure, part of it is ego. Armed with hindsight and (perhaps) superior tools and technologies, most systems look like they could be built differently. And armed with a specific type of hindsight - the type that tells you exactly where the weaknesses are - can be intoxicating.

But the reality is a bit harder to handle.

Hindsight has blindspots:

The tools and techniques might be newer, but are not always better

I’ll let Marianne Bellotti explain it:

The first mistake that software engineers make with legacy modernization is assuming that technical advancement is linear. With that frame of mind, anything build in older design patterns or with an older architectural philosophy is infererior to to newer options. Improving the performance of an old system is just a matter of rearrnaging it to a new pattern.

My experience dealing with Frankenstein systems like the one described has taught me that progress is not linear. It’s cyclical. We advance, but we advance slowly, while moving tangentially. We abandon patterns only to reinvent them later and sell them as completely new.

The classic example of this in recent times is thinking through what a mainframe is and then thinking through what an AWS managed service is.

You might be operating with less constraints than they were

The first version of the system was likely built in one of the following scenarios:

• A startup in which the system should be built in the next few months so that the company does not go under, and it should pretty much work at launch, then very quickly evolve based on feedback

• A rewrite of another system - someone was given permission to rebuild a working system and had to do it quickly, and their reputation was on the line

Both of these situations are different than your current one (most likely).

Respect should be given for birthing the system in the first place

Bringing something from a non-working state into a working state is the ultimate performance improvement. Sure, you might be tuning some “god-awful” queries that never existed before. That 2k line stored procedure that you like to make fun of used to be nothing and had to be made quickly and correctly to solve something very real. Like screenwriters like to say: “Where the f were you when the pages were blank?”. You should respect bringing nothing into something as much as you criticize something.

You are only seeing the final version of the system, not what else was tried

And that 2k line stored procedure likely grew into 2k lines, starting out in a much simpler form. You are looking at version 8 or 18, and working on version 9. Maybe version 4 might have been a complete rewrite of something that didn’t scale at all. They may have tried the things you are proposing as obvious solutions and they did not work.

Respect your Elders, Respect Yourself

From Amazon’s Principal Engineering Community Tenets:

Respect What Came Before

Principal Engineers are grateful to our predecessors. We appreciate the value of working systems and the lessons they embody. We understand that many problems are not essentially new.

Key phrase: appreciate the value of working systems and the lessons. If you write off the existing system as made by fools, you don’t appreciate the value that it pretty much works and the lessons that this brings. You turn off your ability to learn from these two critical aspects of an existing system.

With a respectful attitude, you are more open to learning. If you view those who came before as a parade of idiots, you won’t learn from them. There are always a number of reasons things are the way they are. Some of these reasons are no longer valid, and some are. Do not tear down load-bearing walls.

An Architectural Decision Record is a document that explains the active tradeoffs when a design decision is made. These might exist for the system you are working on, but a worthy exercise is to try to create one and see how far you get. This forces you to seek more information and learn from what is in front of you before you start to tear it down. And, of course, build one for changes you make and for new people coming onto the project later.

Use it for Good

The feeling that you are better than the current system can be a good positive force in small quantities:

• It makes you feel that you can improve things

• It makes you feel that the system is more easily changed and improved than maybe it is

• Everyone else thinks it can’t change, so a little irrational hope goes a long way

Weaponize your enthusiasm and ignorance to push for improvements, but respect those who came before and seek to understand why and how they built the system you are working on.

---