Organizational Vignettes: How to Conduct a Knowledge Management and Learning Needs Assessment?
From microLINKS Wiki
In their own words… Each partner provided an analysis of their own process and experience with knowledge management and learning needs assessments, specifically highlighting the elements of those processes that worked and didn’t work within their own organizational contexts.
Contents |
CARE
CARE implemented a KS&L assessment of the six southern African countries participating in the project. To incorporate the most state-of-the-art practices and processes, CARE collaborated with Accenture Development Partners (ADP) to assess the current state of knowledge sharing with respect to the VS&L program in six selected countries and to identify barriers to effective learning. This assessment identified the key needs to enable effective learning – in terms of human capacity, policy & staff incentives, and technology – and evaluated options for addressing these needs, drawing on CARE’s experience in the field as well as external leading practices in the areas of knowledge management and organizational learning.
The needs assessment was based on a knowledge sharing framework consisting of nine components that can be applied to a program or organization. This framework was intended as a flexible approach that could be adapted by CARE to assess KS&L opportunities in other programmatic contexts. The nine components of the knowledge sharing framework are:
- Strategy: the comprehensive plan to be implemented for managing organizational knowledge.
- Governance: the structure and systems by which an organization controls and oversees its knowledge processes along with best practices and the organization's strategic vision.
- Monitoring and Evaluation: the methods to measure ongoing knowledge sharing activities and their link to organizational productivity.
- Culture and Behaviors: the norms and attitudes toward knowledge sharing activities in an organization.
- Policy and Incentives: the understanding of what compels knowledge sharing behaviors in the organization and the alignment of those behaviors to performance management practices.
- Technology / Tools: the enablers of knowledge sharing behaviors and activities that accelerate time to competence and productivity within an organization.
- Content: the content architecture and set of standard requirements to enhance content quality in an organization.
- Communications: the channels of communication used to deliver and share knowledge in an organization.
- Process: the standard work processes to support the systematic sharing and use of knowledge in an organization.
In order to understand the current knowledge sharing situation and desired future state, structured questions within the above nine framework components were developed and data collected from the relevant stakeholders within program (see appendix C for examples of questions). The analysis of findings revealed the current state of knowledge sharing and learning within the program as well as gaps for each component.
What worked: Investing in the needs assessment and framework proved to be beneficial for the development of a knowledge sharing strategy and implementation plan. With an assessment being conducted before energy is invested in building an effective learning organization, everybody understands what is at stake, what the problems are and what types of rewards can be achieved. This will help achieve wider buy-in and participation at all levels.
The needs assessment and framework findings allowed CARE’s Economic Development programming to focus initial energies on the areas with the greatest opportunity for change, since not all the areas can be attacked for various resource constraints. To make the most impact and gain more KS&L “champions” focusing on the areas with the greatest opportunity for change for your program or organization according to the assessment framework results will end up making large strides to building an effective learning organization;
What didn’t: Although the culture and behavior component was not found to be an area for the greatest opportunity for change, it is important not to underestimate the investment needed to strengthen perceptions of knowledge sharing culture and behavior. Underlying all of the practices and processes that will be put in order to carry out the steps to building an effective learning organization are new behaviors and culture of knowledge sharing. For example, if an electronic database is created for the repository of knowledge, the behavior of gathering and storing knowledge at the repository must enter the habits of the stakeholders. Placing emphasis on enforcing the necessary behavior and culture for learning to take place will enable the focus on building the KS&L components to occur.
Freedom From Hunger
Freedom from Hunger used the BELO project funding to improve KM at the organizational level. The initial needs assessment was rather straightforward, and asked staff what types of information they wanted improved access to. This preliminary assessment helped to scope the types of information that should be made available on the organizational portal.
After about a year of working on the portal and introducing knowledge management to the organization, it became clear that there were other issues impeding knowledge flow. Consequently, the Knowledge Manager decided to conduct a more comprehensive organizational knowledge audit. This audit consisted of three parts. First, staff was surveyed on human systems related to knowledge sharing which included learning style preferences, motivation and incentive sources, comfort with technology and their experiences with the portal to-date. Second, various categories of knowledge assets were identified within the organization as well as the factors that prevented people from accessing them. Knowledge asset categories included documents such as reports, meeting notes, etc., education modules, email, databases, contact lists, staff expertise, etc. These knowledge asset categories were arranged in a pyramid so as to see how one form of knowledge influenced another – for example: a document is written by a person who performs a given set of roles within a process which is influenced by the organizational culture (see knowledge pyramid in appendix C). From this, the knowledge manager identified trends and root issues to be addressed by the knowledge management strategy. For example, Freedom from Hunger has experienced rapid growth and a great amount of organizational change during the last three years. By looking at where documents were stored, how electronic records were used, processes that people followed, who went to who for help and even the way people described Freedom from Hunger, it became evident that there were two distinct staff groups, a smaller group that had been there for a long time and a larger new group of staff that had been recently hired. Because many of the old processes were not documented, new staff did not know about them and therefore developed new systems and processes. Similarly, the old staff members continued to follow old processes not realizing that anything had changed. A strategic issue was therefore to decide which set of processes to follow and how to get everyone on the same page. Third, a knowledge workshop was held with staff to teach them more about knowledge management so that they could identify more thoroughly their own needs and priorities.
These three data sources painted a very cohesive picture as to what was going on in terms of knowledge sharing and information flow within the organization and gave us clear priorities and direction. Once work started to address the priority issues, even more detailed analysis of artifacts, processes and human systems with each issue was needed.
What worked: The quickest and most effective means of assessment was the knowledge workshop. People are very capable of identifying what works and what needs improvement. It was found that by doing a more participatory assessment people better understood what knowledge management was and why it was important. They were also better able to see how individuals and teams are part of a larger system and how changes in one place can “break” things in another.
What didn’t: While the knowledge workshop was one of the most effective and efficient assessment tools, it wasn’t fully polished when rolled out to staff. The lesson learned is that assessment tools should be tested with a small but diverse group of people to ensure they are easily understood and the logistics of rolling out the tool are fully thought through.
While the knowledge management workshop was one of the most effective and efficient assessment tools, it took a few iterations to perfect. The initial goal of the tool was simply to teach staff about knowledge management and to get them to reflect on what they might be able to do to improve their knowledge sharing within their project or team. During the workshop, staff was able to provide a great amount of feedback, which the knowledge manager was not prepared to formally capture. Fortunately, most of the feedback mirrored the conclusions of the knowledge audit and supported the KM strategy. Moving forward, however, the knowledge manager has realized the value of the feedback given during these sessions and has uses it as a way to more formally facilitate the development of both team and organizational knowledge management plans.
Practical Action
Practical Action's needs assessment experiences occurred in a programmatic context: the Markets and Livelihoods Program. This meant that senior program staff (i.e. team leaders in each of the countries where Practical Action works) knew the needs of their own team members and processes relatively well.
The Markets and Livelihoods Program explored the following tools to assess KS&L needs:
Learning Map: This is a tool in which key learning processes are mapped out in a graphic fashion (similar to a flowchart). This tool promoted rich analysis and dialogue between staff around the routines, stakeholders and logistics involved with learning. For example, it allowed the UK team to learn about learning routines that are important in the country offices but invisible at the UK level simply because they were created by the local teams according to local needs and culture. However, this tool has at least two downfalls:
- Its scope is limited to learning processes and does not question the broader level of the learning strategy and principles.
- It is difficult to make comparisons between learning maps produced by different teams (the map has to be accompanied by a report or narrative that explains it in detail. This narrative is difficult to compare to others).
Learning Routines Matrix: This tool maps out key learning routines and their most important features. It does not rely on graphic representations (like the Learning Map), which tend to be complex where there are many interrelated learning moments. Instead, it simplifies the format to gain in comparability between country teams and efficiency of information collection. The Markets and Livelihoods Program developed a matrix that proposes frequencies (daily, monthly, yearly, occasional, etc) in the vertical axis and key features in the horizontal axis (description, audience (internal/external), type of reports produced, and degree of perceived importance). The matrix also promotes useful analysis of key learning moments. The degree of ‘perceived importance’, using a star rating-system, was particularly useful, especially as it highlighted learning routines that were perceived in different ways by various country teams. A good balance in identification of key features (horizontal axis) is critical: leaving important features out can exclude relevant information from the needs assessment and including too many can make the process inefficient and disappointing.
Learning Model M&E Matrix: The Markets and Livelihoods Program is moving towards a more comprehensive approach of learning M&E (which can be understood as a cyclic needs assessment). Monitoring how key organizational learning processes are evolving and paying attention to the impacts of those changes is fundamental if Practical Actions wants to become a true learning organization in a systematic and more effective way. Appropriate M&E of organizational learning processes allows for better planning and cost benefit analysis.
Since the beginning of the BELO project the program team defined four learning loops that are still relevant and fundamental for its practice. The matrix has not been applied across the whole team yet, but will be validated by the team leaders during the next International Team Meeting.
- Loop 1: reiterative learning within the program (including the UK and country/regional teams)
- Loop 2: reiterative learning among market system actors (e.g. producers, buyers, service providers, processors, policy-makers, etc)
- Loop 3: reiterative learning between market system actors (with special emphasis on the most marginalized ones) and the organization
- Loop 4: reiterative learning between the organization and other organizations working in the field of pro-poor market development
Peter Senge’s Learning Model [1] suggests a set of critical aspects for effective organizational learning; namely, guiding ideas; infrastructure; theory-methods-tools; attitudes-beliefs; awareness-sensibilities; and skills-capabilities. Practical Action is currently developing a matrix that crosses these two models (our 4 learning loops and Senge’s) to establish a map of concrete needs and aspects that each country/regional team will focus on during a given time. It is still too early to tell how useful this matrix will be, but it should help in tracking improvements in learning processes, highlight weak and strong areas of organizational learning, and identify areas of potential international experience sharing (e.g. if a team makes progress in an area of the matrix where another is weak).
WOCCU
WOCCU underwent during early 2007 an internal knowledge management (KM) audit, with the goals being two-fold: first, to determine and ascertain individual understanding of what a “learning organization” is and what KM brings to WOCCU; and second, to determine what is already in place at WOCCU in order to increase capacity to share knowledge throughout the organization. The summary of the KM staff audit was presented to senior management in advance of a senior manager meeting. The senior level meeting focused KM methodologies on specific areas/topics in which WOCCU’s best practices and lessons learned documentation could promote replication in other countries. Additionally, the meeting identified the need for a required reading list for all staff to ensure they shared the same basis for understanding WOCCU’s work.
What worked: By conducting a series of individual meetings with senior managers via the KM audit, there was a broad-based approach to sensitizing senior level decision makers as to the idea of knowledge management. Meetings with senior managers would be followed by staff meetings to present the KM position and the organizational strategy and explain where the organization is headed with KM, learning, and training. This process would also serve to clarify and disseminate senior management vision, policies, and priority goals throughout the organization so that managers and staff better understand where and how they “fit” into the larger picture.
What didn’t: The buy-in from senior managers as to how KM is most suitable within the WOCCU context did not occur in one standard, uniform manner. This caused confusion and limited success with the deliverables as there were a variety of expectations.
Proposed set of next steps:
- Establish and lead a KM advisory group and inter-departmental knowledge management team.
The advisory group would oversee the implementation of WOCCU’s organizational KS&L strategy and help obtain senior management and staff buy-in and commitment. This team would seek to improve communication and collaboration throughout the organization, among and between departments, and among and between senior management and line managers, and staff. The advisory group might meet quarterly (or semi-annually) and communicate via conference call, emails, and Skype, to make certain the learning, training, and KM needs of staff and organization are being addressed. Ad hoc knowledge management teams could be formed to address specific learning or knowledge management needs.
- Improve outcomes and communication at WOCCU meetings. Set up a standard way of reporting and tracking information from meetings, so that it is relevant to staff and details accountability and follow up.
- Research and develop a set of materials and learning service providers to support staff training, development, and e-Learning with the assistance and collaboration of HR. Concentrate on supporting the development of training and learning materials for “New Hire” orientation or for a technical staff certification program. Identify external sources of training and e-learning appropriate for staff—this avoids the need and expense of WOCCU developing its own courses when they already exist elsewhere.
- Conduct KS&L needs assessments involving HQ and field offices to identify learning, sharing mechanisms and barriers to enable to start mapping what needs to be addressed. More importantly, the audit will allow WOCCU to start mapping out the processes currently in place and simultaneously evaluate the knowledge life cycle within the organization. The assessment also addresses: a) When, where, and how information/knowledge is created? b) How is institutional knowledge captured and saved; c) Is staff aware of how and where knowledge is stored?
Notes
- ↑ Pasteur, K. (2004) Learning for Development: A literature review. Lessons for Change in Policy and Organizations No 6. Brighton: Institute of Development Studies.
