The stories told in Narrative Assessment are owned by the interviewee as the teller of the story. They are highly personal, sharing perspectives and accounts of events as they unfolded in advocates’ working lives. It is the interviewee who is to decide how much is told and with whom. Since sharing is the main purpose of Narrative Assessment, the stories must be safe enough for interviewees to allow for sharing.

Narrative Assessment asks for a wealth of details and some of these may make it quite easy to identify precisely who did what where, why, and with what effects. This information may be useful to those who oppose the goals of advocates and programs. For example, if an advocate successfully uses a family network to build trust with a senior official, and if this senior official acts in the way the advocate suggested, public recognition that this official's actions were influenced could put the advocate, the official, and the programmatic gains at risk. Further, circumstances may change over time so stories that were once safe later become dangerous. These changes are likely not to be detected by outsiders. As such, those who share stories must know that they own them. This means they have the right at any time, to have their contribution modified or removed.

A three-legged stool

Evaluation often seeks to establish relationships between outcomes, mechanisms (how the outcomes were achieved), and context. Outcomes assessment is, predictably, interested in outcomes. The standard of validity for outcomes assessment is transparency: it should be possible to confirm precisely what is reported. Narrative Assessment focuses on the other two parts of this three-legged stool, namely, mechanisms and context. These two other parts are often used for a different purpose: improving practice. By gathering stories, Narrative Assessment builds a rich understanding of how mechanisms contribute to outcomes and the role of the context in this.

An insightful, plausible story showing how or why things happened as they did does not require full transparency. In those cases where full transparency is considered necessary, it is possible to gather information that helps others in a manner that does not specifically describe actual practice or an actual context, in a way that can make people or actions involved identifiable. This section describes a few strategies that can be used to improve the ability of people to report useful information, preserve the integrity of the data gathered, and protect the people and programs that have shared their stories. 

Safe gathering and reporting

Facilitators should not ask for or hold information that could be used to harm. If it is not possible to talk about things as they actually happened, then the facilitator and interviewee, before starting to record their session, may agree on a strategy that the interviewee will use to hide those parts that create risks. The simplest way to hide the origin of a story is to change details like the names of the organizations involved, dates, and locations in the story in ways that blur the connection to specific events, actions, or relations. This can also be discussed at the closing stage of the interview or afterward.

If this blurring is not sufficient, an alternative approach is for the facilitator to ask the interviewee to present their narrative in the third person (‘I have a friend who…’), to depersonalize the account by explaining how somebody in a situation like theirs might respond, or to change details in the story that are not relevant to the lessons learned.

While this may sound as going against the transparency and verification purposes of sound evaluation, it may make sense for advocacy and thus Narrative Assessment, since an important purpose of Narrative Assessment is to understand how things happen in advocacy. It is possible to tell a valid and useful story using a fictional setting. The decision to fictionalize parts of a narrative must be taken consultatively as they must both protect and communicate. While the person asking for permission to do so need not reveal the details, they must provide a strong justification. Incidentally, this sort of fictionalizing need not alter the nature and level of detail present in a narrative.

But what about context then? An accurate description of the exact context in terms of people, places, and exact events may not be crucial for building understanding (of, for example, why an unusual strategy succeeded). What matters is ensuring that the details provided support accurate interpretation. It is possible, and sometimes necessary, to modify details about context in stories and have distant readers still make correct interpretations. This means that heavily modified stories can support valid assessments.

To make it possible for a team to validate or more deeply address the story, the facilitator, with permission of interviewees, may choose to create a document stating the modifications, and store this securely.

Categorization, storage, and retrieval

To be able to use stories for various purposes, it is important to gather and store them, and to do that in ways allowing for access and retrieval of different types of stories (e.g., stories about outcomes, stories about a certain part of the program, or certain types of challenges). This is particularly helpful in cases where large numbers of stories are produced, as with evaluations involving multiple organizations or several countries.

Categorization

Stories will be structured around themes. These themes, which will be informed by program interests, will become thematic tags. In addition to these thematic tags, interviewees will answer the question ‘what is most important in the story you just shared?’. This will create a list of ‘key points’ tags. Each narrative will also be classified according to a standard set of variables like location, program, date, the identity of the interviewer, and any other non-thematic variable that is relevant. All three kinds of tags will be put as keywords at the top of the story.

Storage and retrieval

Recordings of interviews, original language transcripts, and all working documents (including an ‘info’ file that contains all relevant information about the story) can be stored in a folder accessible only to the team directly involved in that narrative assessment. This folder is to be categorized as confidential.

The stories produced from interviews, edited for safety and confidentiality, can be stored both in the original language and in English.

Once the stories, both in original language and in English, have been checked to ensure they do not create risks or violate legal requirements regarding personal data, they can be placed in a folder within the ‘Narrative Assessment stories’ folder that is accessible to others involved, for example, staff of other organizations involved with the same program. All these stories the consortium must be safe for public access.

To facilitate retrieval, the location of each story, tags, and contact information can be entered in an Excel sheet stored in the ‘Narrative Assessment stories’ folder. Management can decide whether to keep or destroy confidential records at the end of the project.