This blog is a continuation of Parts 1 & 2.
Throughout this 3 part series, I’ve discussed how the ServiceNow knowledge base can resemble the prisoners’ dilemma, bringing about selfish actions that may weaken the effectiveness of the knowledge base as a whole.  I’ve suggested multiple ways to curtail this effect, including several organizational change management (OCM) efforts.  However, all of these OCM efforts can be rendered meaningless if the quality of the knowledge base starts to drop.  Bad content leads to selfish users in the knowledge base.

 

Bad Content Leads to Selfish Users

Imagine a user with a question who two choices – either going to the knowledge base, or emailing around until he finds the correct answer.  Emailing is the selfish action – it will save the user time, but take up time of the people he emails.  Instead, let’s pretend that the user decides to go to the knowledge base, but finds it full of outdated, inaccurate information.  The user is going to say “Why did I bother looking in the knowledge base, anyway? It’s full of junk!”

How likely is it that this user will use the knowledge base the next time a question comes up?

Knowledge Admins – The Garbage Men

As a knowledge admin, it’s important to make sure that authors understand this effect of bad content.  Bad knowledge articles don’t just negatively reflect on that article – they can sour users’ perceptions of the entire knowledge base, making users less likely to use the knowledge base in the future.

In an unchecked environment, this can result in a negative feedback loop.  If users start using the knowledge base less, authors may see a disincentive from writing good knowledge articles – “People don’t check the knowledge base, so why bother writing a good article?”

As a knowledge admin, it’s important to make sure that authors understand that a bad article doesn’t just reflect poorly on the author, it reflects poorly on the knowledge base itself.  To keep the knowledge base running smoothly, knowledge admins need to be the garbage men of the knowledge base – disposing of the trash, and cleaning up the sloppy articles.

Kaizen your Knowledge Base

One way to clean up the knowledge base is to kaizen the knowledge base – have a culture of continuous improvement around your knowledge base.  One way to implement the kaizen is to force users to improve sloppy content.

In one sense, this could be thought of as the opposite of “rewarding good efforts.” (link to relevant article) If you force an author of a sloppy article to rewrite the article, you are essentially punishing the sloppy writing habits.  This will continue to fix the misaligned incentive structure (link to relevant article) – if the author has to spend extra time fixing a bad article, then maybe it would just be quicker to write it well in the first place.

But more than that, most sloppy articles are not a product of intentional negligence.  A low-quality article may simply be a situation of an author not realizing the standards of a good article.  Also, time can cause article quality to decay.  An article can become inaccurate or obsolete if it stays in the knowledge base for too long.   Like we mentioned above, poor quality articles can destroy users’ faith in the base itself; thus, it is imperative to keep reviewing content to ensure that articles aren’t degrading over time.

How to Improve Poor Content

At my Fortune-500 consumer packaged goods client, we have used a number of methods to maintain high quality articles.  Every article goes through two reviews before being published in the knowledge base to ensure that articles are high quality at the get-go.  First, they are reviewed for style and grammar by the knowledge admin, and then a subject matter expert performs a technical review.

Once an article is published, the author makes use of the built-in feedback tools to help identify articles that may have slipped past the initial review or degraded over time.  The knowledge admin regularly checks the feedback of articles to identify flagged articles that need to be rewritten or clarified.  Articles that are flagged as inaccurate have an email automatically generated to the author to initiate an immediate review.

Finally, all articles are required to be reviewed by the author at least once a year to prevent articles from becoming outdated.  The yearly review is initiated using the valid to date.  The author can set the valid to date for up to one year in the future, and the author setting the valid to date is his way of verifying the article is currently accurate and will be continue to be accurate until that date passes.    When the valid to date gets within a month, business rules automatically fire an email to the author, reminding him to review the article and reset the valid to date if the article is still valid.  The act of resetting the valid to date becomes the author’s way of certifying the article is accurate.  If the article is no longer accurate, the author is encouraged to either edit it as necessary or retire it.

If an author ignores the emails, the article will go past the valid to date, and no longer be visible in the knowledge base.  The emails, and the consequence of the article being no longer visible, gives the author the incentives to actually complete the yearly review – often good habits like this one can slip by the wayside if there are no consequences.

Because the article can become invisible in the knowledge base, it’s important for the knowledge admin to monitor for articles that have passed their valid to date, and validate whether the author simply forgot to review the article, or if the article is actually outdated and needs to be retired.

Knowledge – Energy That Feeds on Itself

Hopefully it’s clear that there’s a certain energy that the knowledge base has that feeds on itself.  When left alone, this can be a negative energy where authors and readers can be incentivized to “betray” the knowledge base by putting in low levels of effort.  This negative energy can incentivize other users to slack as well, causing a negative feedback loop where the usefulness of the knowledge base is continuously declining.

It’s up to the knowledge admins to inject positive energy to ensure the knowledge base functions smoothly.  As a knowledge admin, you are the garbage man of the knowledge base.  You need to make sure the knowledge base stays clean – that junky old articles are either polished up or retired. By injecting positive energy, you can create a positive feedback loop that ensures the knowledge base is running well.

By ensuring good buy-in, rewarding good efforts, and implementing systems to improve poor content, this can tip the scales in favor of using the knowledge base correctly.  The same way that the negative energy can feed on itself, the positive energy can feed on itself, incentivizing both authors and readers to use the knowledge base to its full capabilities, and saving everyone involved time and energy in solving issues.  With this injection of positive energy, you may find less need to police the knowledge base, as people will be fixing their articles as needed.  Ultimately, by injecting this positive energy, you can help keep your knowledge base clean as a whistle.

 

Note: This article has also been posted to my company’s insights page.

ServiceNow Knowledge Base can be like the Prisoner’s Dilemma. So shift people’s perspectives of the game.
This blog is a continuation of Part 1.

In the previous blog in this series, I discussed how the ServiceNow knowledge base can resemble the prisoners’ dilemma, due to a bad incentive structure, and discussed ways to change the incentive structure to help users make choices that benefit the knowledge base as a whole.  However, there’s an underlying assumption in the metaphor that users are only self-interested, and this selfishness is part of what brings about the poor incentive structure.  With any faceless technology, there’s always the threat of users losing sympathy for the other users.  As a Knowledge Administrator, it’s important to make sure that users are aware of their other users.  By making users sympathetic to their fellow users, the misaligned incentive structure may evaporate entirely, solving the prisoners’ dilemma scenario without needing to directly change the incentive structure at all.

How are People Self-Interested?

How Prisoners are Only Self-Interested (Prisoners’ Dilemma POV)

Let’s refer back to the prisoners’ dilemma analogy from the previous post. The model above assumes that the prisoners are taking an “every man for himself” approach to their plea deals.  There is no sense of comradery or honor between the two prisoners that might encourage them to risk betrayal to help the other one out.  And conversely, the prisoners aren’t going to feel any guilt if they betray the other one, causing the other additional jail time (man, these are really some ruthless criminals).  Thus, when the criminals make their decision to betray the other, they are only thinking of their own outcomes – they don’t care about the outcome of the other.

How Users Are Only Self-Interested (ServiceNow POV)

Similarly, in this theoretical knowledge base, the participants are only self-interested.  By writing a sloppy article, the authors reveal that they are not respectful of the readers’ time.  Similarly, the readers are not respectful of the authors’ time by emailing them directly, rather than searching for a knowledge base article.

How might this happen in a real knowledge base scenario?  Are we assuming that all knowledge base users are just jerks who only care about themselves?  Is this model realistic for a real-world knowledge base?

Probably not to the full extent of this example.  However, it’s possible to imagine a scenario where users don’t think of their fellow users as much as they should.   With any faceless software, it’s easy for users to forget the other people behind the scenes.

How Can Knowledge Admins Deter Self-Interest?

As a Knowledge Administrator, it’s important to recognize that the knowledge base incentive structure may resemble the prisoners’ dilemma, encouraging users to behave selfishly.  However, if you can change users’ attitudes to think about other users, the prisoners’ dilemma situation may disappear entirely, since users will now act with other people in mind.

Good User Buy-In

Ensure that both authors and readers understand the importance of doing their part to the best of their abilities.  Using effective Organizational Change Management (OCM) techniques can help users understand the importance of all the different roles in the Knowledge base, and why it’s important for everyone to do their part effectively.

“What’s in it for me?” – Make Sure People Know the Correct Answer

With good user buy-in, users can answer the question “What’s in it for the knowledge base?”  But more important than that, is making sure that users can answer the question “What’s in it for me?”  As a Knowledge Admin, it’s important to make sure that users understand the reciprocal nature of the knowledge base – that by doing their job correctly, it makes others’ jobs easier as well, and vice-versa.  By understanding their own personal benefit, it will be much easier to create a lasting change.

Envelop High Standards Within the Company Culture

Being selfish is the lazy way to use the knowledge base.  As a knowledge admin, it’s important to set a culture of expecting high standards within the company.  Point out examples of the high standards of the company, and encourage users to be proud of that culture. If people feel like they’re a part of something bigger than themselves, they will be likely to go that extra mile to help maintain those high standards.

You’ve Users Interested in Others – Now How Do You Keep Efforts High?

Now that you have users thinking of the entire knowledge base, everything’s well and good, right?  Maybe not.  Imagine this scenario – a user is excited about maintaining the high standards of the company, and searches the knowledge base for his answer, even though it might take longer than emailing his coworker.  When he searches the knowledge base, he finds it littered with outdated, and factually inaccurate articles.  “Why did I bother searching the knowledge base, I had to email a coworker, anyway. Next time, I’ll just email the coworker directly.”

In the next article in my series, we’ll talk about using Kaizen principles to continuously improve your knowledge base, to help keep users thinking about other users in the knowledge base.

Note: This post has also been posted on my company’s insights page.

ServiceNow Knowledge Background

ServiceNow is a Platform as a Service (PaaS) that offers an array of Service Management applications, with roots in IT Service Management (ITSM).   Among them is a knowledge base, where users can gather, analyze, store, and share knowledge with the purpose of improving efficiency by reducing the need to rediscover knowledge. [1]

Prisoners’ Dilemma Background

The prisoners’ dilemma is a classic thought experiment that shows an example of where people might not cooperate, even though it is in their best interest as a group to cooperate. In the classic example, there are two prisoners who are being interrogated for a crime. Each prisoner has two choices – either turn in their partner for a reduced sentence (betray), or stay silent (comply). For this example, we’ll say that if both prisoners comply, and stay silent, then they both will only get a 1 year sentence. If they both betray each other, then they’ll each get 3 years.   But if only one prisoner betrays the other, then the prisoner who betrays will be released, while the prisoner who stayed silent will get 5 years. Each prisoner has to make the choice independently and without discussing with each other– they can’t collude to both stay silent. The chart below summarizes the options and payoffs:

Implications for ServiceNow Knowledge

Clearly, the prisoners would be better off as a group if they could both stay silent –

there would only be 2 total years of prison time. However, this scenario tends to end with both prisoners betraying. A double betrayal is what’s referred to as the Nash Equilibrium, which essentially means that this is the most likely outcome (see the link for more information).

Fortunately, the stakes are much lower for ServiceNow knowledge bases. No one is going to prison (at least, I certainly hope not!) However, the basic structure of the prisoner’s dilemma can potentially apply to the Knowledge base. Using the prisoners’ dilemma as a framework, I’ll discuss some aspects that can lead to a lower quality knowledge base and specific examples of how to fix these issues.

Imagine that instead of prisoners, the two parties are the authors of articles and the readers of the articles. The parties have two choices, similar to the prisoners’ – they can either “cooperate”, or “betray,” meaning that they can either do something beneficial to the knowledge base, or something more selfish. For the author, this would be either writing a high quality article that includes some of the attributes previously described, or writing a sloppy article.   For the reader, the choices would be either checking the knowledge base for the answer, or just emailing the author for the answer. Rather than prison time, the outcomes are measured in the total amount of time that the person has to spend for his portion of the knowledge base. Again, in this thought experiment, there are no outside agents, and the author and readers make their decisions in a vacuum.

How is This Similar to The Prisoner’s Dilemma?

Though the numbers are different, this thought experiment is structured in such a way that the Nash equilibrium is the same as before – with both parties betraying each other.

Why does this happen? There are several problems that lead to a Nash Equilibrium outcome of a double betrayal: 1) there’s an incentive structure that discourages cooperation, 2) the parties are only interested in themselves, and 3) there’s a lack of trust between them.

I’ll discuss reasons 2 and 3 in future articles, but for now let’s examine reason 1 – the incentive structure – from the prisoner’s dilemma and see how it applies to this Knowledge situation.

Misaligned Incentive Structure (From Prisoner’s POV)

If the warden was attempting to create an incentive structure that would encourage the prisoners to betray each other, he got it.

The reason for this is that they will individually get less jail time by betraying, regardless of what the other prisoner does. For example, if prisoner A stays silent, prisoner B will go free by betraying, but would get 1 year by staying silent. Or if prisoner A betrays, then prisoner B only gets 3 years by also betraying, versus 5 years by staying silent. In both scenarios, prisoner B is better off by betraying, and the same thing happens for prisoner A. This leads to the Nash Equilibrium outcome of both prisoners betraying.

Misaligned Incentive Structure (from ServiceNow POV)

The incentive structure in the ServiceNow Knowledge base is similar to the prisoners’ dilemma – the optimal solution to minimize total time involved is for everyone to cooperate and do the best job they can (write a good article, and check the knowledge base for answers).

Similar to the prisoners’ dilemma, the author has the incentive to write a sloppy article because regardless of what the readers do, he will spend less total time if he writes a bad article (if the readers check the knowledge base, he would spend 1 hour total on a good article, versus 50 minutes total on a sloppy article). Similarly, the readers have an incentive to just email the author, since they will end up spending less time total by doing so, regardless of the quality of the article that the author writes (1 hour and 20 minutes on the good article side, and only 50 total minutes in the bad article case). Assuming that the time estimates are reasonable, the incentive structure leads to the same Nash Equilibrium from before – with both parties betraying. Regardless of the specific time estimates of this example, it’s easy to imagine a scenario where both parties would have an incentive to slack.

How Can Knowledge Admins Avoid Bad Incentive Structures?

In the classic prisoners’ dilemma, the incentive structure is set up in such a way that can incentivize users to act in a way that’s detrimental to the overall health of the base. It’s important for Knowledge Administrators to recognize this pitfall, and ensure that appropriate steps are taken to remove these temptations. I’ll discuss two ways to fix this bad incentive structure – altering the incentive structure, and changing users’ perceptions of success.

1.      Change the Incentive Structure

This might seem obvious, but the best way to prevent users from falling into this pitfall is to change the incentive structure to reward the behavior that makes the base work properly. You can do this by rewarding good efforts, and making sure that poor efforts have to be corrected.

Reward Good Efforts

Make sure that authors and readers are properly rewarded when they do a good job. These rewards need to be on-going to make sure users continue to perform their duties effectively.

Rewarding good efforts can help fix the misaligned incentive structure. In the example above, the only rewards are in terms of time spent looking for the solution, or time spent writing an article. Additional incentives can be created to help motivate users to maintain a high quality knowledge base.

An example that a knowledge admin could implement would be recognizing authors that consistently produce high quality articles during yearly reviews. Quarterly rewards (giftcards, etc.) given to the authors with the highest rated articles would also help encourage good content.

Implement Systems to Improve Poor Content

This could be thought of as the opposite of “rewarding good efforts.” If you force an author of a sloppy article to rewrite the article, you are essentially punishing the sloppy writing habits. This will continue to fix the misaligned incentive structure – if the author has to spend extra time fixing a bad article, then maybe it would just be quicker to write it well in the first place.

How to Improve Poor Content

As far as improving poor content, CapTech suggested a number of strategies to a Fortune 500 client. Stay tuned for a future blog post about improving low quality knowledge articles!

2.      Change Users’ Perceptions of the Game

The other way to change the incentive structures is to change authors’ perceptions of the incentives. The assumption we’re using above is that users are only thinking about how to minimize the time that they individually spend in the knowledge base.   But what if you can change the users’ perceptions, and have them not be only focused on their own time, but of the total time spent in the knowledge base. This can make a huge change to the fundamental incentive structure. I will discuss how to make this shift in Part 2 of this series, which will be released soon.

Don’t Like the Outcome? Change the Game.

Throughout history, entrepreneurs have succeeded by changing the rules. When it comes to ServiceNow Knowledge bases, the rules of the game can be stacked against you. As a Knowledge Administrator, it’s important to recognize that the incentives in the ServiceNow knowledge base may be stacked against you. But you have the power to change the rules, and alter the incentives to encourage people to use the Knowledge base in a productive manner.

In the next post in this series, I’ll discuss how to shift users’ perceptions of the incentives of knowledge – to get users to think of the entirety of the knowledge base, not just themselves.

 

Note: this blog has also been posted on my company’s insights page. Check them out for more information and insights on ServiceNow and other technologies!

Sources

[1]: http://wiki.en.it-processmaps.com/index.php/Knowledge_Management