Showing posts with label metrics. Show all posts
Showing posts with label metrics. Show all posts

2010-09-16

Value for Knowledge Workers

Demonstrable value goes a long way to supporting the deployment of new software tools.

For structured business processes, return on investment (r.o.i.) is comparatively easy to estimate. Where unstructured or semi-structured digital content items (e.g. documents, spreadsheets, faxes, etc.) enable a given structured process (e.g. accounts receivable) their contribution to the overall value created is also typically quantifiable.

Where the process itself is unstructured the measurement of value is much harder. Perhaps the largest class of unstructured processes in a company fall in the category of knowledge work. The difficulties organizations have in understanding knowledge work is highlighted in an article just published in the McKinsey Quarterly entitled: "Boosting the productivity of knowledge workers".

  • Aside: Unfortunately a subscription is required to read the full article – hopefully you have one.

The article starts with the proposition that few senior executives can answer the question: "Are you doing all that you can to enhance the productivity of your knowledge workers?" This is unfortunate because, "Organizations around the world struggle to crack the code for improving the effectiveness of managers, salespeople, scientists, and others whose jobs consist primarily of interactions—with other employees, customers, and suppliers—and complex decision making based on knowledge and judgment."

The authors, Eric Matson and Laurence Prusak, describe five common barriers that hinder knowledge workers in more than half of the interactions in surveyed companies:

  1. Physical
  2. Technical
  3. Social or Cultural
  4. Contextual, and
  5. Temporal

Physical barriers include geographic and time zone separation between workers, and are typically linked to Technical challenges – where workers lack the necessary tools to overcome the physical barriers that separate them. As the article notes, there are a many software tools available that can help – these would include the various collaborative and social media tools, as well as the more classic document management applications that are encompassed in the broadest definitions of Enterprise Content Management (ECM).

Of course the availability of software tools does not guarantee that users will use them effectively; indeed, Social (e.g. organizational restrictions, opposing incentives and motivations) and Contextual barriers (e.g. not knowing who to consult or to trust) play a large part in hindering adoption.

The fifth barrier is Temporal. Time, or rather the perceived lack of it, is also a critical factor. In my experience knowledge workers do not consider time spent using social media and collaborative tools as important as other activities. Under time pressure they will stop using these tools if they need to spend more time on other activities they perceive as "real work".

What struck me in reading the article is that while an increasing proportion of staff in companies are knowledge workers, it is clear that what knowledge work is and how to best enable it to drive productivity gains is not clear. Given that, it is hardly surprising that people struggle to define the value of those software tools best able to support knowledge management.

2010-08-12

Content Matters

I was chatting with a colleague yesterday and he related how he interviews people to join our company. We quickly dropped into role playing – with me as the job candidate. He had a compelling proposition, but as I told him, he was missing the thing that excited me = Content Matters!

As I started to tell him why content matters I found myself getting excited. I realized I'm actually quite passionate about it! Not content itself, but what it enables and how it's used.

Content matters to companies in a way that changes how they work, how they create value and whether they succeed. It matters whether they recognize that fact or not.

If you want to understand what drives a company look at their value chain – how they create value – and how they are organized to execute each stage in the value chain. Within each stage there are typically many processes, each with many steps. At almost every step there is some content that is created, reviewed, followed or otherwise used; how well this is done makes a difference to effectiveness.

It's no surprise to people that you can understand a business by 'following the money' or 'following the customer' and that is the basis for ERP and CRM systems.

On the other hand most people are only just coming to realize that 'following the content' is just as important, so while we've talked about content management for many years, that conversation is starting to be important to business.

2010-06-11

Considering the Cost & Value of Digital Content for an Enterprise


The way that the value of digital content changes over time, and how an enterprise content management (ECM) system might help to realize and/or retain greater value was the subject of my last post (http://martin-fulcrum.blogspot.com/2010/06/calculating-value-of-content-in-ecm.html).

Lee Dallas retweeted that post, but also referenced a very interesting earlier blog post (2008) by fellow member of 'Big Men on Content' Marko Sillanpää
on the cost of content (link). Sillanpää considered content lifecycle costs as follows:

Cost of Content = (Annual Authoring Costs + Annual Review Costs) / New Objects per Author
Content authoring and review are not the only activities that incur cost – there are costs associated with each step in its lifecycle, notably including the costs of distribution, storage and ultimate destruction. Effective content distribution is becoming increasingly important to the realization of value.

Cost and value are of course different concepts. The cost of an item does not necessarily reflect its value, as anyone who has watched the TV show "Antiques Roadshow" knows!

In business, where there is an emphasis on the bottom line, the value of content ought on average to exceed its cost, or it should not have been created. But for a given piece of content, its cost is generally related to size and complexity, not what it enables. On the other hand, value is tied to enablement and varies over time – often declining gradually or precipitously, but sometimes increasing!

It can be hard to explain to people how managing content benefits a business. However, I have found that identifying its 'enterprise value' is powerful. A good top-down approach is to reference the value chain of a business, using Michael Porter's original simple model.

People understand that enterprises take input from suppliers and partners and, through a series of steps, add value that can be realized in a final sale to customers. Clearly the effective execution of those steps adds to efficiency. When challenged, most people can identify content that contributes or is even essential to the completion of each of those value steps and their constituent processes. For example, an Engineering Department must create, review and approve engineering drawings, and then pass them on to the Manufacturing Department (see E, C & O value chain).

In my experience, taking a value perspective is generally more attractive, especially in growth industries, than a cost and cost avoidance perspective – which has classically been the basis for return-on-investment (R.O.I.) approaches to software justification.
 

Syndicated at http://conversations.opentext.com/

2010-06-04

Calculating the Value of Content in ECM


It's only worth expending effort to manage something if it has value – usually positive, but sometimes negative. So the concept of content value is implicit in enterprise content management (ECM).

On the other hand, the value of a given content object (i.e. digital file) such as an email or document generally declines over time – or at least this is the common wisdom. I have seen graphs drawn mapping 'value' over 'time', with a smooth decline of value tending to zero. However, such a representation is clearly an average of value across many types of enterprise content.

If you look at individual pieces of content, then you'll find different profiles:

  • In compliance, a piece of content may retain 100% of its value for a defined period of years and then abruptly drop to having no value, or even having negative value (liability) that should trigger its destruction
  • In knowledge management, a piece of content may have declining value over time, but then because of some new event may suddenly have increased value
But this perspective is of the Inherent or Independent Value of a piece of content – the value is assessed entirely based on the information contained in the object. But it seems to me that there are at least two other factors that impact value:

  • Context – when correctly combined with other prices of content a given piece of content may have greater value. For example a specifications document is more valuable together with the associated requirements document. Value can often be realized by the way in which context is presented between content items – how they are grouped, ordered or ranked.
  • Impairment – Ironically, the value of a piece of content may be impaired by efforts to manage content. If you mix valuable pieces of content with large amounts of irrelevant materials, that should have been destroyed, you reduce the chances that the valuable content can be found and its value realized. Keeping everything is usually a bad idea. And often users impair value when they misclassify content.
So the available value of a piece of content to an organization may be expressed as follows:

Available Value = Inherent Value x Context / Impairment
What this says is that content management efforts can be beneficial, but if not done well can actually be destructive.

2009-12-18

Customer Community Success Metrics for 2009

Customer communities are all the rage nowadays, but it is not always clear what works and indeed how to measure success.

As 2009 draws to a close we have been reviewing how Open Text customer communities have been doing.

Background: For those not familiar with Open Text, we are a vendor of enterprise-class software to manage digital files (called content). The term enterprise indicates that we sell to organizations not consumers. We have relatively few customer organizations, but they are often typically some of the biggest organizations in business and government. We estimate that at least 1 in 3 Internet users visit sites that run our software! The software we use for our own communities is the same as we sell.

The Sites

For historic reasons, we run three primary community sites (requiring membership) in addition to our typical corporate websites. The community sites are:


  • Open Text Knowledge Centre (KC)
    • Primarily for system administrators of the software we sell


  • Open Text Developer Network (OTDN) which is housed on the KC
    • Primarily for developers using Open Text APIs


  • Open Text Online Communities
    • Primarily for business champions and power users

Site Metrics

  1. The Knowledge Centre is by far the oldest community, dating back to 1996! As you'd expect, it has the most members and the most ongoing activity. Every day approximately 4,000 users access the site, and between 150,000-200,000 documents downloads are performed every month!
  2. OTDN just completed its first full year during which just over 3,200 unique users participated over the past year
  3. Online Communities got started in its present form in 2005. This last year 10,600 members collectively visited 118,000 times over the year
These numbers only measure direct participation. As you might expect, many community members participate through email-mediated discussions.

Convergence

Multiple systems have traditionally meant that there are multiple, disconnected silos of information. As a result, users don't know where to look and administrators have to duplicate critical content between systems.

A better approach is to deploy a single, 'enterprise library' of digital files (content) which contains all of the files, but just one active copy of each. The three sites above will soon converge to use the same enterprise library, which will also be used by our corporate website that is open to the general public.

One single repository can make user navigation harder unless the most relevant content is presented and organized in a fashion that best meets the needs of each type of user (i.e. persona). Communities of users with similar interests or jobs are one approach to organizing content, but of course there are others, including personalization based on the activities and preferences of specific users.

Measuring 2010 success

These communities will continue to develop, but the latest social networking approaches provide new ways to surface important content. As we deploy more social networking approaches during 2010 we'll have a solid base of community metrics from 2009 to judge progress. As you might expect, activities on external sites like twitter, YouTube and facebook are becoming increasingly important.


Syndicated at http://conversations.opentext.com/

2009-07-14

Official vs. unofficial social networks and their impact on user adoption

Guest post from my colleague Agnes Kolkiewicz originally posted to Open Text Online Communities Metrics community (Metrics Blog).





In my last post, we looked at the correlation between the success of collaborative systems and user adoption. Many of you struggle to obtain user adoption within your organizations for communities of practice (CoP). We mentioned that staged deployments are usually recommended by analysts and that the resultant 'success stories' can then be showcased within the company to further promote user adoption and justify the investment. But where can you find a good initial 'use case' within your company that is likely to succeed?

Consider the following excerpt here from a post by Etienne Wenger, an internationally recognized expert in the field of learning theory:

"A community of practice is different from a team in that the shared learning and interest of its members are what keep it together. It is defined by knowledge rather than by task, and exists because participation has value to its members. A community of practice's life cycle is determined by the value it provides to its members, not by an institutional schedule. It does not appear the minute a project is started and does not disappear with the end of a task. It takes a while to come into being and may live long after a project is completed or an official team has disbanded." (http://www.co-i-l.com/coil/knowledge-garden/cop/lss.shtml)

It has long been observed that unofficial networks exist within a company. People join these because they see value in participation. Knowledge is being exchanged within those networks outside of any organizational structure, crossing information silos. When designing an information system that supports communities of practice and collaboration, the challenge of an administrator may very well lie then with initially identifying what unofficial networks already exist within the organization and designing systems that will support these networks.

You can start by asking yourself the following questions:

  • What is important to our employees?
  • Where do they find value?

What might follow then is that a community of practice should not be imposed on a group of people according to organizational structures – rather, it should serve as support for unofficial networks that have already been created within the company. The value to the organization exists then because the knowledge that is already being exchanged through informal discussions, IMs, or email is now being captured and archived within a community of practice, and is then easily searchable by newcomers to the organization or people outside of the specific community that also have an interest in the subjects being discussed. This type of exchange is much more difficult to recreate when establishing a community of practice based on official organizational structures.

This is not say that communities of practice should not adhere to organizational principles further down the line, but chances are that by identifying such unofficial networks within a company, and supporting then using social media, you might stumble upon an initial 'success story' that could be then used to promote the technology throughout the organization.

2009-06-24

Obtaining and Using Key User Adoption Metrics for Enterprise Communities

Online communities are an appealing but challenging topic for enterprises. Most organizations feel they will increasingly use social networking tools, but are uncertain how best to proceed.

Proving business benefit and responding to deployment challenges can be a difficult, but these activities must be grounded on objective data, so obtaining key metrics is essential. However, often community and social networking applications intended for enterprise deployment lack the necessary query and reporting tools to support business champions and system administrators.

Recently we launched a Success Metrics community on Open Text Online Communities for those who have deployed Open Text Communities of Practice (CoP). Members of that community have access to reporting tools and templates specifically designed to provide community metrics.

My colleague Agnes Kolkiewicz just made the following blog post in that community (see Metrics Blog):

 
 

Many of you have mentioned that you are struggling with user adoption within your communities. As I believe that the success of any software project (and particularly where document management and collaboration are concerned) is largely dependent on user adoption, I believe it would be within the scope of this community to include some best practices on user adoption. To support this, I will start a "user adoption best practices" wiki in the community with the hope that all of you can add to it as we progress.

Metrics and user adoption, especially in the case of collaborative or social media software go hand-in-hand. The more that substantive content is added and the higher the participation and log-in rates, the more value your system will bring to the organization, but you need to be able to measure that progress and growth in participation:

  • Measuring log-in and participation ratios will enable you to determine how your efforts to improve user adoption are progressing. These ratios will also enable you to identify "peak times" in participation - this way you can likely pinpoint what had caused the surge in participation and aim to recreate the event. By measuring these ratios on a regular basis, you will also be able to demonstrate at a later date how the system has improved over time and set a foundation on which the overall value of the system can be determined.
  • Metrics are applicable throughout a project lifecycle. This applies to any software project but it can easily get out of hand where large enterprise deployments are concerned. This is why most analysts recommend that you start off with smaller deployments and apply metrics to measure progress. Once successfully deployed, you can then use the success of this project, and any ROI realized, to champion the solution as a "success story" throughout your enterprise to further promote user adoption in subsequent projects.
  • You should start looking at metrics in the pre-project phase taking into consideration what your objectives are for the software project at hand: these objectives will help you identify metrics as well as KPIs for the deployment. The developed metrics should then be measured throughout the lifecycle of your software deployment until the software is no longer used by your organization.
  • In many cases, the same metrics that were used to evaluate the overall health of your system will also form the foundation for any later cost-benefit or ROI calculation. Where no metrics are kept during deployment, it is much more difficult to "go back in time" and determine how the project has progressed over time and identify realized improvements as there is no benchmark for comparison. We have tried to overcome this obstacle by creating some reports that allow you to measure certain aspects of your deployment at a specified date, but that hasn't been always possible. A rule of thumb here is to start thinking of metrics as an integral part of any project you undertake.

I would encourage everyone to use metrics throughout your project lifecycles – please keep in mind that metrics not only help you prove an ROI, but also help guide your project as it progresses by enabling you to measure progress and identify areas that work well and those that need improvement. In the case of document management or collaborative software, user adoption goes a long way in helping ensure the "success" of your project.