Organizational Lore: How It’s Costing You and What to Do About It

by

old-books-436498_640

I define Organizational Lore as the sum of all the nuggets of undocumented knowledge within a company that are passed along by word of mouth. Most organizations have some and many have critical processes that rely on it. Often ignored, Organizational Lore can be extremely costly to your operations and left unchecked can pose a real risk to your business.

Story Time!

A company that I’m familiar with sells a decision support tool that runs off a custom database. Each customer has their own database and the data is received from a variety of different sources. New data must be standardized and loaded into the customer’s database regularly. The company has a complex data management operation where Data Analysts are responsible for:

  • Verifying the incoming data quality
  • Transforming the data (including writing the data transformations)
  • Verifying the quality of the transformed data
  • Loading it into the customers database, which can be heavily customized depending on the type of data being stored
  • Quality checking the database load using their customer-facing application

The Data Analyst job requires a broad range of technical and analytical skills that must be learned on-the-job. There are manuals, however no one could pick up the manual and do this job. Why? Because the manual describes the most “vanilla” case, and in reality every client has their own “flavor”. Someone, likely the person who worked on that customer previously, has to be available to explain the quirks, answer questions, and help with the inevitable troubleshooting. Does this sound familiar? How successful do you think this company is at consistently producing high quality databases to support their products?

The cost of Organizational Lore

The effects of Organizational Lore in these situations can manifest in several ways:

Unpredictable quality and an inability to recover quickly

Mistakes are going to happen when your processes depend on Organizational Lore. Recovering from a failure is harder because no one is familiar with your exact “flavor” of the process. Product Support tries to help, but is less effective and has to escalate more often. Quality suffers and your customers lose faith in your ability to deliver reliably.

Steadily growing complexity

When a problem came up at the example company they would change their standard process to prevent the problem from happening again. They would add steps, create checklists, or add stopping points in the process for review from a senior resource. The result, however, was a frustratingly slow and complex process that took more work to complete and didn’t improve quality.

Turnover trouble

The burnout rate is high for jobs like the Data Analyst in the example above. Not feeling secure in your ability to do your job is stressful, particularly for new employees. Worse, when you lose senior people you lose the part of the lore that was stored in their head, which leads to….

The loss of “why”

“Why do we have to do this step?”

“I don’t know, we’ve always done it that way.”

Over time processes can evolve beyond the bounds of institutional memory. People know they have to perform a step, but they don’t know why. This makes it very difficult to troubleshoot problems or make operational improvements. No one wants to break the system!

Diminished security

It’s hard to secure something you don’t understand and holes can be buried in mounds of complexity. It’s as simple as that.

A silver lining?

Is there a positive side to Organizational Lore? Possibly. Processes that are hard to learn are even harder to duplicate. If your business relies on this kind of a process it could be a competitive strength. However, process that rely on Organizational Lore tend to get more complex over time and a new competitor might be able to create a new process without all the baggage and your advantage would quickly become a disadvantage.

What to do about it

If you’re in a situation like the example company it’s not as hopeless as it might seem. There are things that you can do to get back on the right track:

Simplify!

Processes that evolve over time can get huge and messy. Special cases and work-arounds accelerate this. It’s a good practice to reexamine why things are done the way they are and to consider alternatives. And, don’t forget to listen to the people that actually do the work. They live the process every day and will have important thoughts on how to fix it.

Insist on documentation and watch for smokescreens

Although hopefully rare, I’ve seen instances where Organizational Lore has been created intentionally to provide “job security.” Be open-minded and pay attention if someone tells you that they can’t create documentation because the process is too complex. It may very well be true, so you need to assess the process and related technology to understand what’s going on. But if it’s not true, you may have a knowledge hoarder that needs to take part in some mandatory cross-training.

Move the lore from your people to your technology

Often times Organizational Lore compensates for deficit technology. If the system can’t handle a special case, then the people in operations have to work around it by passing around lore, creating their own tools and utilities to fix problems, etc. This is okay in the short-term to avoid an operational slow down, but fixing the technology should be a high priority.  Remember, much like Technical Debt, you can also create Operational Debt. The time your people are spending on fixing problems and creating workarounds is taking away from the time they should be spending on operational efficiency. Let the technology do the hard work and allow for more standard processes for your folks in operations.

Organizational Lore can be a tough problem to solve, but with the right analysis you can find the technological improvements and process efficiencies you need to make it right.

 

READ MORE

Fake Case Study: Jack of all trades vs. Master of One

Fake Case Study: Jack of all trades vs. Master of One

  Listen to any earnings call or executive presentation and you will likely hear the terms “top line” and “bottom line.” These are words used to describe a business’s performance. According to Investopedia, the words are defined as follows: Top line refers to the...

read more
Your Personality Is Showing

Your Personality Is Showing

There I was, minding my own business one evening, digging into my organization's SEO performance (as one does), when I came across something interesting. Search terms related to "MBTI" — or the Myers-Briggs Type Indicator, developed by Katherine Cook Briggs and Isabel...

read more
Lessons From a Change Manager Who Hates Change

Lessons From a Change Manager Who Hates Change

Hello. My name is Monique, and I’m a change manager who hates change.   After years of receiving “consulting therapy” from various mentors, I am now able to say these words out loud and proudly. But for a long time, it felt more like an admission of guilt. I mean, who...

read more
Creativity as a Cure

Creativity as a Cure

The topic of creative solutioning has been front and center these days as we talk more and more about organizational adaptability in the face of dynamic and uncertain times. For example, I recently read about a project that got me thinking about specific priorities...

read more
Thought Ensemble, a Pariveda Company — Why Now?

Thought Ensemble, a Pariveda Company — Why Now?

Big news over here as we close out the year - we have been acquired by Pariveda, a 750-person consulting firm in 12 markets across North America! We are now “Thought Ensemble, a Pariveda Company” and I’ll be serving as the Managing Vice President continuing to lead...

read more
Thought Ensemble Joins Pariveda Solutions!

Thought Ensemble Joins Pariveda Solutions!

Dallas, December 9, 2021 /PRNewswire/ -- Pariveda, a leader specializing in solving complex technology and business problems, announces the acquisition of Thought Ensemble. With the addition of Thought Ensemble, Pariveda now provides holistic business strategy,...

read more
Thoughts on Colorado’s Equal Pay for Equal Work Act

Thoughts on Colorado’s Equal Pay for Equal Work Act

It was about a year ago that we first started hearing about Colorado’s Equal Pay for Equal Work Act (SB19-085) and I knew it was going to be national news. We’d just gotten past the “Rocky Mountain High” jokes, and our lovely state was trying to break new ground...

read more
Disruption Is the New Normal

Disruption Is the New Normal

By nature, disruptors are not popular. “First they ignore you, then they laugh at you, then they fight you, then you win, then they copy you.” We have all heard some version of this quote, and we have all seen it play out in real life. We've seen it with building...

read more
What Would You Say You Do Here?

What Would You Say You Do Here?

“I deal with the … customers so the engineers don't have to! I have people skills!” That famous Office Space quote from Tom Smykowski cracks me up every single time. I know Toms. I’ve been Tom. Change the quote to say, “IT Team” instead of “engineers,” and there’s a...

read more