Notes

[Design and implementation – lead in]
Performance Measurement Systems are poorly designed: When designing a performance measurement system, it is important to correctly identify the major stakeholders and how they will be affected by management decisions, so that the common goal is correctly established. Even if management can accurately decide on their goals and priorities, there is also the problem of choosing the suitable model: there are many theoretical frameworks for these systems, but little on their application and practical limitations. The implications of the models and usefulness of metrics also have to be considered – or it can lead to various negative consequences, including failing to negate principal-agent problems and rewarding the wrong behaviors – which can be surprisingly common. This is extremely important: when designing performance measurement systems: there is often a tendency to rely and fixated on metrics. Because metrics provide a large incentive for managers to follow, the emphasis on metrics can often lead managers to disregard common sense in favor of decisions that will lead to better measurements. For example, an automobile factory might reward managers when production quotas are met while failing to make the manager account for overtime costs, quality or storage.
Performance Measurement Systems are poorly managed: Most commonly, a poorly managed performance measurement system will lead to inaccurate data that does not reflect actual performance, because managers often have an incentive to make the reports look good for them. An example is that a delivery company may measure on-time delivery by considering only that the products leave their inventory at the specified time instead of having the products delivered to customers on time. An effective performance measurement system also needs to be frequently revised and changed for different purposes and even business periods. Therefore, they also need to be continually re-evaluated and audited, but a change in performance measurement system often leads to huge changes in other systems, which can be both costly and difficult to implement.
Performance Measurement Systems may fail to meet the needs of users: Very often, organizations can become too focused on measuring and reporting performance that measures lose their meaning and become ineffective in real decision making. Performance and Quality departments are often too focused on collecting as much data as possible, generating many measures and reports, while most fail to reflect the link to performance or perceived as pointless for decision making. Sometimes, these systems may provide the wrong measures with the right purpose in mind. This can happen when the system is designed in a way that rewards best practices without participation from the providers and users of reported metrics. For example, a call center may measure the wait time for customers, which lead to its staffs trying to end calls quickly in order to reduce the wait time for customers in queue, disregarding the quality of the calls and whether customers concerns are properly addressed.
Lack of organization capacity: A performance measurement system can be complete in theory, but it will not function properly in practice without capacity of the organization. A good system will require organizations to properly allocate sufficient resources for collecting the data, auditing and managing systems properly. For small-scale organizations, a less sophisticated system may be more desirable for practical applications.
[identifying external factors – lead in]
Execution:
[Data is not relevant or useful – lead in]
Data collection costs are too high: Generally, a performance measurement system is as reliable as the data it is based on. It is often considered good practice to invest on data collection and measurement. However, it is also important to note that performance measurement systems also lose their value if their operating costs outweigh their benefits. Because the actual effectiveness of a performance measurement system can be difficult to accurately evaluated, it might be more practical to take a reserved stance and opt for a simple and cost-effective system.
Data validation and audits are inadequate: As much as there are strong incentives for managers to make measures look good on reports, this can affect the way data is collected or even cause managers to report falsified data. Data collection can also easily be affected by expected human error. There must be sufficient investment and effort put into data validation and audits to make sure that the data is reliable and measures taken actually reflect the state of the organization at any given time.
Data is of poor quality: Most often, because of human error or the nature of the data taken, it might contain too many errors or be presented in a state that makes it difficult to derive any meaningful measures from. This usually happens with text or content-based data, which is normally difficult to standardize, quantify or categorize. It is usually mitigated by imposing restrictions on inputs or automating the data collection process. Data also has to be collected in a timely manner with respects to the needs of the business for it to be actionable. This usually means having a schedule for updates to account for any deviation, or simplifying the data collecting and reporting process. Finally, data must be communicated along with its scope – users have to be aware of the process by which the data is obtained or calculated, and everything that any particular data element is comprised of: a prime example would be the “total revenue” element.
[Performance information utilization – lead in]
Performance reports are unusable: Aside from problems with too many irrelevant measures and data as mentioned above, measures must also be presented in reports in a way that is easily understandable to management. Important points must be properly highlighted without bias in a manner that makes it easy to derive the proper and desirable follow-up actions. In practice, performance reports are often fixated on presenting too many graphs and charts that may become distractions and also give rise to biases – focusing on the less important but optimistic data while not emphasizing enough on the more actionable data.
[Performance data is not used effectively]
[Feedback mechanisms are inadequate]