Saturday, March 30, 2019

Fixing the Broken

I was reading my twitter feed, when I stumbled upon this:
If a process is broken throw it in the trash and start over. Nothing is set in stone.
The simplicity of the tweet is absolutely true.  It totally reminded me of a problem I've seen multiple times though.  The process is rarely the difficult part of fixing a problem.

A long time ago, when I was a team lead, the group I worked with had an automated build system that was extremely complicated, built entirely in-house, and didn't follow the conventions of any of the standard build-systems that exist.  There was a steep learning curve to get new software packages into the system, and most developers never learned to do it.  This also meant that when someone needed to introduce a project that didn't exist already, they'd often work-around the need instead of waiting for someone with knowledge to help.

At some point, the group hired a published tech-author who was a big open source advocate, and quite outspoken.  Upon trying to interface with the build system, they loudly declared it broken, and suggested it should be fixed.  Management above me said a very smart thing, "Okay, fix it."

Many months later, that person left the group, and the same build system was still there and there were not even any modifications done.  Let me unwind what went wrong.

By most metrics, if a process is that hard, then it is objectively broken.  That new employee was absolutely correct in the assessment.  However, the process doesn't care.  This group had over 200 distinct, but interrelated projects, which means that any replacement system would need to be configured for all of them.  The process also had a custom syntactical structure to deal with a number of edge cases.  This is both what made it maddeningly difficult to work with, but also what made it work well in that environment.

The process, like most, was built and maintained by people, most of whom were still sitting in that office.  Generally, everyone who understood the custom build process deeply appreciated the many, many things it accomplished.  Those who didn't need to learn the complexity of adding a new project didn't have a reason to care, but that also meant that those people weren't useful allies, since they didn't understand the full extent of what the process did.

This new person basically started by angering the very people who maintained this process.  Those were the only people with the knowledge of the complexities that a new system would need to mitigate or replace.  Once this person started working in earnest to replace the system, the complexities showed themselves randomly, and the original maintainers just stood back, waiting for the inevitable failure.

Here is the lesson I learned from watching this happen.

To affect change, you must acknowledge a process' power, and demonstrate understanding of its complexity before the process guardians will trust you to replace it.

If a process accomplishes nothing, it would have already been replaced.  If a process exists at all, it accomplishes something, and it is probably there for a reason.  Sometimes a strange or ugly process simply exists because of somebody who did something really dumb.  As in, don't be the person who makes us write a rule.

If a process is terrible, ask open questions about why it is built the way it is.  Most likely it is still there because it solves problems that other tools (even if they are newer) don't already solve.  This is especially true when those processes bridge multiple other systems.  Avoid criticizing a process, but ask pointed questions about the parts that are ugly.

On multiple occasions in the many years since, I've been able to use these lessons to fix or replace multiple processes in multiple places.  In no case has the problem been a technical hurdle, but a problem of finding those who protect the process, and getting them on-board with fixing the deficiencies (even if they don't need to do the fixing, but just stop being protective).  Sometimes this means a whole process replacement, but most often it has meant paying off technical debt (like major code refactoring or updating dependent systems), and implementing new interfaces into the existing processes.

Wednesday, March 20, 2019

[Book] Mortal Engines by Philip Reeve

Book cover
Over a thousand years before the book's present, there was a war that effectively destroyed all of society.  Picking up the pieces of the technology that was left behind, London was put onto treads, run by steam, so that it find and consume other towns for resources and, ultimately, more fuel to keep moving.

Over time, other towns and cities did the same, while another group, called the Anti-Traction League, created a defensive wall across the only pass in a mountain range to keep these traction cities at bay on the other side.  This gives the background of how steam is the primary driver while other technology - well past steam - is also present.

By putting steampunk over 1000 years into the future, this book represents one of the best thought-out steampunk universes I've ever read.  The social constructs around living on - or avoiding, predatory cities are incredibly well thought out.  That is, this presents as a plausible future.

The story is mostly told from the perspective of Thomas, a young Londoner who is a "Third-class apprentice at the Museum of London".  The hero of this story is a young woman named Hester, who is trying to avenge the murder of her parents.  The first action of this story is Thomas preventing Hester from killing her target, and subsequently falling off of London with her.

In the best science fiction tradition, this ornate, fantastical background is a perfect set-up to reflect our present back on ourselves.  At its core, this book is about learning to accept that heroes may be false and that a character's society, itself, might be built upon evil.

The story is incredibly well done, though there are a few parts where the dialog between people is a little flat.  The masterful pacing and very well described action more than makes up for the places where odd dialog momentarily bumped me out of the story.

If you plan on reading the book and seeing the movie, I recommend watching the movie first.  The stories hit the same core-points, though the movie is much more of an action adventure than well balanced action and story.  For me, at least, the book fills in large amounts of detail, some of which was hinted at through movie dialog.

If you like Sci-Fi or Steampunk, I highly recommend this book.  Trigger warnings for slavery and extreme violence.  Skip it if you prefer fiction to be in the known world.

Saturday, March 9, 2019

Managing Difficult Problems

I can no longer count the number of times that I've been able to re-invigorate a problem investigation, even if I have zero visibility on the actual problem.  This takes some self-discipline that doesn't come easy, especially during an urgent investigation.

Here's that one weird trick:

I do the depth of reading myself.  If I see multiple threads, I'll read all of them.  Then I will write as short a summary of all of the facts that I can.  Re-summarizing all of the relevant facts that have been shared, pointing out the places where multiple people or departments have a differing view of the facts, and sometimes suggesting a list of questions that should be put back to the customer (the person who is reporting the problem), will usually refocus and reinvigorate the investigation.

Wait what?

When something goes wrong, e-mails have a tendency of getting very long reply chains as people add a few sentences and add more people who might be able to help.  This is pretty normal, and isn't actually a terrible way to go about finding a problem solution.  The urgency is obvious, so most people just skim the top-most e-mails, and keep the chain moving.

On a normal day, few people will read e-mails beyond about two pages worth of text (some report as little as a paragraph).  During a difficult or urgent problem, depth of reading is not likely to get better.  I'm not here to lament this, it is just a fact about humans.


I started doing this back when I did product support (so long ago it doesn't even hit my resume anymore).  It came from a place of wanting to be able to contribute even when I didn't know the answer myself.  Sometimes by writing the summary, I would be able to see the actual problem and just answer with a solution.  Most often, though, the questions I would come up with would lead directly to a solution.  Frankly, it might be one of the things that I did that helped others think that I should be a manager.

Now, as a manager, I know that I am rarely going to have the answer, so it seems natural to continue doing the depth of reading and actually contributing back a summary and a few questions.  That is, to me, the very act of trying to write a summary of a problem naturally leads to important insights into a problem.

Problem space

It would seem that the people who have been on the thread since the beginning would be annoyed at seeing all the things they already said be repeated.  This has happened twice that I know of over the last 20 years.  It has never happened when the summary also brings up a disparity of reported facts.  In any case, I've taken to explicitly starting with a line similar to this, "I am summarizing this thread to clarify my understanding of what is going on here, and to introduce the problem to those recently added."  I also find it very important to end with something like this, "If I have anything wrong, or I missed an important detail, please let me know."

Every time I've done this, it has led to immediate changes.  First, it is a point where a large number of people can legitimately leave the investigation (even if they can just start ignoring the thread).  That is, some folks who know they have nothing to do with the problem are literally only hanging on to make sure that their one piece of input was heard.  Especially in cases where I am pointing out a dispute in the facts, a number of people will re-investigate the dispute.  About half of the time, the problem itself lies within the dispute.


Please ask questions if you have them.  Also feel free to let me know if there's anything above that I should add.  I wish to improve this if I can.  After some time, I'm likely to republish this on LinkedIn.

[Movie] Won't You Be My Neighbor (2018)

Movie poster
This is a documentary film about Fred McFeely Rogers, who was on a popular children's program called Mister Rogers' Neighborhood from 1968 through 2001.  The movie starts his career with a children's show that he produced before Neighborhood, the Children's Corner, though skips his earliest work for NBC.

Won't you be my neighbor is a very well paced, carefully timed, and beautifully edited documentary.  It took a relatively small number of interviews with people from his show and life, a healthy portion of interviews from Mister Rogers himself, and a lot of footage from his shows along with just a touch of vacation footage.  I learned a lot more than I thought I would.

Watch this movie if you grew up watching this show.  Watch this movie if you want to know and understand the link between Public Broadcasting, Children's programming, and Fred Rogers, specifically.  Skip it if Mister Rogers' is nothing more than a meme for you.  Not everybody watched it, and a lot of recent adults never got a chance to watch it.

Wednesday, March 6, 2019

[Book] Fear by Bob Woodward

Book cover
I haven't been sleeping well.  A good friend of mine suggested that my reading this book may be one of the reasons.  I can't dispute that directly.  As I write this, right before New Year's 2018, I'm actively looking for employment, and that is stressful, but this book definitely hasn't helped.

Subtitled Trump in the White House, Fear is about the presidency of Donald Trump and written by the Washington Post reporter Bob Woodward.  The book is separated into 42 numbered but untitled chapters which are mostly organized by time, though not entirely.  Each chapter reads like two or three long, stand-alone articles about some set of probably related events or issues during the early parts of Trump's presidency, though I often missed how or why each is grouped together in the same chapter.

The book is well written, and seems to take great pains to state things that happened without editorializing.  This is old-school reporting, which is a difficult style to get used to again, now that I've been living in the modern age of commentary-instead-of-news.  There's a lot of stuff described here.

Emotionally, though, I found the whole book to be exhausting.  Sometimes, because I disagree with what Trump did or tried to do.  Sometimes, because I disagree with things people on Trump's staff did to prevent Trump from doing something.  That is, regardless of whether a reader is a Trump supporter, there's a lot to feel exhausted about.

I almost didn't have the energy to finish this book, and there are a lot of valid reasons to skip it; emotional stamina being near the top.  I read the paper regularly, so there wasn't a lot of major events recounted here that were new to me, though the described machinations of how the White House runs under Trump was enlightening.  Read this book if, like me, you are a politics junkie.