A ForecAsting guide For new & underused Methods oF FAMily

A FORECASTING GUIDE FOR

NEW & UNDERUSED METHODS

OF FAMILY PLANNING

What to Do When There Is No Trend Data?

A FORECASTING GUIDE FOR

NEW & UNDERUSED METHODS

OF FAMILY PLANNING

What to Do When There Is No Trend Data?

and Population Services International: Washington, D.C.

Recommended Citation:

Institute for Reproductive Health, Georgetown University (IRH/GU), John Snow

Inc. (JSI), and Population Services International (PSI) for the Reproductive Health

Supplies Coalition (RHSC). 2012. A Forecasting Guide for New & Underused

Methods of Family Planning: What to Do When There Is No Trend Data?

Washington, DC: IRH/GU, JSI, and PSI for the RHSC.

Acknowledgements:

The conceptualization and documentation of this guide was a coordinated effort

among Georgetown University’s Institute for Reproductive Health (IRH), John

Snow Inc. (JSI), and Population Services International (PSI), with input from key

experts in the field of global health.

The core writers and reviewers of this guide, by partner organization, include

IRH—Meredith Puleio, Victoria Jennings, Donald Cruz, and Marie Mukabatsinda;

JSI—Ellie Bahirai, Paul Dowling, Carolyn Hart, Trisha Long, Joseph McCord,

Meaghan O’Keefe, and Greg Roche; PSI—Dana Tilson, Maxine Eber, Chattarpal

Chauhan, and Moses Odongo; and PATH—Bonnie Keith.

We would like to acknowledge the following individuals who contributed to the

guide: Katherine Maina, Jonah Maina, Joseph Mburu, Boniface Nienga, Andy

Pillar, Caitriona Rush, Markus Stiener, John Townsend, and Elizabeth Westley. Refer

to Appendix 1 for a listing of the organizations that contributed to the guide.

Support for this project was funded by PATH, on behalf of the Reproductive

Health Supplies Coalition, under the terms of the subgrant no. GAT.1291-05142-

GRT. The views expressed by the authors do not necessarily reflect the views of

the Reproductive Health Supplies Coalition or PATH.

ABSTRACT

This guide provides direction to programs that want to forecast for new and

underused methods (NUMs) of family planning. It supports program managers

and others involved in forecasting as they plan to (1) introduce a contraceptive

technology for the first time in a country, and/or (2) position an underused

method for scale up. The guide recognizes that accurate forecasts take into

account the larger system into which the NUM will be introduced and scaled,

and it offers a framework for building rational assumptions to support accurate

forecasting for NUMs or any family planning method where future demand is

inherently difficult to predict. It also identifies common pitfalls in NUMs forecasting

and recommends strategies to avoid them.

Table of Contents

Acronyms ............................................ 2

Section 1: Background .................................. 3

The challenge and how this guide will help .............. 3

Methodology........................................ 4

Section 2: Context ...................................... 7

The truth about demand forecasting for contraceptives. .. 7

The special case of NUMs and demand forecasting. ...... 8

Section 3: Forecasting for NUMs.......................... 11

Section 4: Common Pitfalls .............................. 27

Section 5: Tips from the Experts .......................... 31

Section 6: Taking the Discussion Forward .................. 35

Section 7: Resources ................................... 37

Appendix 1: Interview List by Organization ................ 43

Appendix 2: Discussion of Variability in MAPE .............. 44

Appendix 3: Data from the 2006-2010 USAID | DELIVER

PROJECT’s Procurement Planning and Monitoring

Report (PPMR) and PipeLine........................... 47

Acronyms

CPR contraceptive prevalence rate

DHS Demographic and Health Survey

ECP emergency contraceptive pill

HMIS health management information system

JSI John Snow, Inc.

ICEC International Consortium for Emergency Contraception

IEC information, education, and communication

IRH Institute for Reproductive Health, Georgetown University

LAPM long-acting and permanent methods

LIAT logistics indicator assessment tool

LMIS logistics management information system

MAPE median absolute percent error

MOH Ministry of Health

MSH Management Sciences for Health

NUM new and underused method

PPMR Procurement Planning and Monitoring Report

PSI Population Services International

RHS Reproductive Health Survey

RHSC Reproductive Health Supplies Coalition

SAM short-acting methods

SDP service delivery point

SID Supplies Information Database

SPA service provision assessment

STG standard treatment guidelines

WHO World Health Organization

Background

The challenge and how this guide will help

Consider this scenario…

The Ministry of Health (MOH) in your country, in an effort to address high unmet

need for family planning, has committed to expanding family planning options

for women and couples by adding a new method to public sector programs

nationwide. The first order of business is ordering an initial stock of the commodity.

When the forecasting committee convenes its annual quantification meeting to

decide how many contraceptives, by method, will be procured for programs in

the coming year, the new method on the list poses a distinct challenge. With no

past usage data to support quantification, how should the committee forecast

demand? Funds are limited, so the committee wants to ensure they don’t over-

order. On the other hand, avoiding stockouts is equally important. How should

the committee approach forecasting for this new method?

How this guide will help…

This guide provides instructions for public- and

private-sector programs that are grappling

with this challenge—forecasting for new and

underused methods (NUMs) (see Box 1) of

family planning when there is limited historical

data. It is designed to support program

managers and others involved in forecasting

as they plan to (1) introduce a contraceptive

technology for the first time in a country, and/

or (2) position an underused method for scale

up.

If a method—such as IUDs, implants,

CycleBeads®, female condoms, or the

emergency contraception pill (ECP)—is new

(or at least new to a catchment area), then

Section

Background

Box 1: What are NUMs?

New and underused methods of family

planning (NUMs) are methods that are

either:

• New to a global or country market, and

currently available for procurement; or

• Underused, as in not routinely available

in the public, private, or social marketing

sectors, and not routinely procured by

the major procurers. In country settings,

underused methods are not present

in that country’s reproductive health

program, despite their presence in a

comparable country’s reproductive

health program.

High quality, effective NUMs can expand

choice in a reproductive health and

family planning program, add value to the

method mix, and respond to the needs of

the clients (RHSC 2011).

historical data may not exist; and, if it does, it may not be useful as a basis for

predicting consumption. This does not mean that programs that introduce NUMs

must rely on guess work. In a resource-scarce environment, there is no money

to waste on over-supply, nor can programs fail to meet their clients’ needs for

suitable methods. While assumptions must be made about procuring the right

quantity of NUMs, they should be informed assumptions.

This guide offers a framework for building rational assumptions to increase the

accuracy of forecasting for NUMs, or indeed, for any family planning commodity

where future demand is inherently difficult to predict. The guide recognizes that

accurate forecasts take into account the larger system into which the NUM will

be introduced and scaled. It also identifies common pitfalls in NUMs forecasting

and recommends strategies to avoid them.

Methodology

To develop this guide, we gathered key informant interviews from over 25

country programs that have been involved in forecasting demand for a

particular NUM. The purpose of the interviews was to better understand

how programs currently address the issue of forecasting for NUMs, given

the lack of trend data; and to gather experience-based lessons learned to

improve forecasting accuracy. Interviewees represented global and local

implementing organizations, including DKT International, FHI 360, International

Consortium for Emergency Contraception (ICEC), iPlus Solutions, the Institute

for Reproductive Health (IRH) at Georgetown University, JHPIEGO, John Snow,

Inc. (JSI), Management Sciences for Health (MSH), Population Council, and

Population Services International (PSI). Project partners also spoke with officials

from several MOHs. The interviews captured perspectives from the following

country programs: Ethiopia, Ghana, Guatemala, Kenya, Liberia, Nepal, Rwanda,

Tanzania, Malawi, Zambia, and Zimbabwe. For a list of interviews completed by

organization and NUM, please refer to Appendix 1.

Additionally, we analyzed data from forecasts done or received by the USAID

| DELIVER PROJECT, and the project’s Procurement Planning and Monitoring

Report (PPMR) from 2006–2010. The PPMR is a monthly report that provides

information on the supply situation of country programs and any short-term

supply issues. From the forecast data, we were able to (1) compare forecasting

error rates of NUMs with more utilized methods (male condoms, oral pills,

injectables); and (2) better understand if the lack of trend data influenced the

forecast accuracy. From the PPMR, we were able to look at the stock levels of

NUMs, over time.

Last, we completed a desk review of existing forecasting guides and tools to

assess how NUMs are addressed, if at all. Most of the tools/resources reviewed

during this process are referenced in Section 7: Resources.

The guide was reviewed by 3–6 staff members in each partner organization (IRH,

JSI, PSI), including four reviewers with field programs. After this review round was

complete, we solicited feedback from members of the Reproductive Health

Supplies Coalition (RHSC).

Background

Section

Context

The truth about demand forecasting for contraceptives

Demand forecasting is the ongoing process of projecting which products should

be procured and in what quantity (Center for Global Development 2007). The

process itself is complex. It requires predicting the quantity of commodities to be

purchased for a country/program, based on need, demand, consumption and

supply.

For contraceptives, forecasts can be based on algorithms and/or simple

calculations that consider a range of inputs, including demographic data (e.g.,

contraceptive prevalence rate [CPR], number of family planning users, unmet

need for family planning); the country’s current contraceptive commodity

mix; consumption data (actual sales and use); financing; program inputs (e.g.,

number of providers trained, promotional campaigns, service delivery strategy);

private- and public-sector involvement and subsequent cost implications for

clients; consumer preferences and willingness-to-pay; geographic scope;

logistics data; service statistics and more.

There is no single “right” way to do demand forecasts. However, some

approaches are proven to work better than others and some data sources

provide more accurate predictions. It is critically important that forecasts be as

accurate as possible in order to provide the number of contraceptive supplies

required to serve the needs and preferences of the population while avoiding

the waste of scarce resources. Demand forecasting is the first step in a much

larger and complex contraceptive supply chain management process that

includes supply planning and procurement; if the appropriate rigor is not applied

during this first planning step, the country’s reproductive health program will face

serious consequences.

It should be noted that forecasting accuracy is highly dependent on the

timeliness, accuracy, and completeness of the data being used. Thus, if the

data are inaccurate, incomplete (e.g., not from all sites), or out of date, the

demand forecast will be affected. Given that NUMs in particular may just be in

the process of being introduced to many sites, or that reporting rates are not yet

consistent, demand forecasting will be a particular challenge.

Section

Context

Most important to underscore is that forecasts inherently will not be perfect.

The USAID| DELIVER PROJECT uses a benchmark of 25% forecast error or less

for contraceptives. That is, a forecast whose median absolute percent error in

forecast (MAPE) (see next section) is 25% or less would be considered to meet a

reasonable standard of accuracy.

The special case of NUMs and demand forecasting

Demand forecasting is particularly challenging for NUMs given the lack of

historical data. And, even if historical data exists, it may not be predictive of

future demand. Thus, current forecasting tools that depend on historical inputs to

forecast demand (see Section 7: Resources) may not be readily applicable for

forecasting for NUMs.

Why is it important to address this challenge? Data from forecasts received

by the USAID | DELIVER PROJECT show that NUMs have higher forecast error

rates than other methods (male condoms, oral pills, and injectables). Forecast

accuracy, or error, is defined as the absolute percentage difference between

projected and actual quantities of a contraceptive distributed in a specific year

for a client or program. Over-forecasting can be determined by subtracting

the quantities forecasted over a specific time period from the quantities

actually used during the same time period. Determination of under-forecasting

is less precise, but it can be identified as an issue if all ordered stock has been

distributed and demand for the product exceeds supply. Forecast errors for

NUMs were more than 53 percentage points higher than other methods, as

measured by MAPE seen in Table 1.

Note the USAID| DELIVER PROJECT uses a benchmark of 25% forecast error or less for contraceptives.

Table 1. Median Absolute Percent Error

in Forecasts by Method Type

2008 2009 2010 MAPE

New and underused methods (NUMs) 50% 145% 36% 77%

Other methods (male condoms, oral

pills, injectables)

22% 17% 32% 23.7%

Difference between NUMs and

other methods

53.5%

Source: Data from the 2006-2010 USAID | DELIVER PROJECT Procurement Planning and Monitoring

Report (PPMR) from 17 countries.

The implication of higher forecast error rates is that NUMs have a greater

likelihood of stock imbalances (stockouts, understocks, and overstocks). The data

show that NUMs—and female condoms especially—have a higher incidence

of over-forecasting. Note that stock imbalances are not solely correlated with

forecast error; they could also reflect other supply chain-related issues, including

financing, distribution, and reporting.

Further, through key informant interviews, we found that most countries do

not approach forecasting for NUMs differently than for other methods, which

probably further exacerbates the forecast error rate, because programs are not

necessarily trying to compensate for the absence of historical data when they

forecast for NUMs. Without historical data, forecasters rely more on demographic

data for assumption building, which often leads to over-estimations.

Refer to Appendix 2 for a discussion on the MAPE variability in Table 1. Refer to

Appendix 3 for a compilation of data from the USAID | DELIVER PROJECT PPMR

from 17 countries, 2006–2010.

A note about variability: Based on the data provided, we see a lot of variability in the overall me-

dianerrorrateforNUMsforthethreeyearsofdatawehave.Bycontrast,non-NUMsshowsignicantly

less variability in forecast error. The quantities of NUMs evaluated were much smaller than for other

methods,whichmayhaveledtohighererrorratesanductuationsfromoneyeartothenext.Also,

the values for NUMs represent absolute derived numbers rather than the median across all countries

and products. For discussion about other possible causes of the variability, refer to Appendix 2.

Section

Context

Forecasting for NUMs

Forecasting for a supply of any method is based on assumptions, but NUMs

are especially dependent on assumptions. This section of the guide explains

a suggested framework for thinking through and building out the assumptions

required for a more accurate prediction of demand for NUMs. Remember that

no forecast is perfect, and a 25% forecast error rate is a reasonable standard of

accuracy.

The recommended process for any NUM forecasting activity includes the

following steps. Each step is explained in detail with recommendations

throughout this section

(USAID | DELIVER PROJECT 2009).

Gather data from secondary sources that can support

assumption building and identify the limitations of the

data

Gather both qualitative and quantitative data. Assumptions should be informed

by data available, including population census data, survey data (Demographic

and Health Surveys [DHS] and Reproductive Health Surveys [RHS]), research

studies, program data about the number of providers trained and number

of facilities equipped to offer the method(s), and any information about the

Gather data from secondary sources that can support assumption

building and identify the limitations of the data.

Build out assumptions based on a contextual framework of factors

that potentially influence the uptake of NUMs.

Host an assumption-building workshop with key stakeholders.

Forecast! And, run a “reality check” on the quantification and

distribution strategy.

Develop and implement a monitoring plan.

Notethattherecommendedstepsforquanticationexerciseswhenhistoricaldataisavailableis

detailed in this reference.

Section

Forecasting for NUMs

experience of the same or like-products (including earlier generations of a

product) in similar markets/countries. Complement this data with information

about how the NUM will be introduced/scaled up in programs for the time

for which you are forecasting—e.g., what are the plans for training, demand

generation, rate of geographic expansion, etc.?

Additionally, speak to program managers, implementing partners, and technical

experts who have experience introducing or scaling up the NUM in another

context. Probe for information that could support assumption building (see step

#2 for tips on questions to ask). If time and funding permit, gather anecdotal

data (or better still, survey data) all the way down the supply chain—it is

important to understand how contraceptives move in the country, including at

the facility and community levels.

Note any inaccuracies and/or discrepancies that may be present in the data

sets. For example, the DHS data may be from five years ago and should be

adjusted to the current situation. Refer to Table 2 to help you think through the

types of data that can be collected to support assumption building and what

apparent challenges may exist in the quality of that data.

Table 2: Types and Sources of Data for Forecasting Demand for NUMs

Type of Data

Sources of Data Challenges in Data Quality

Program background

information

• Policy and strategic planning

documents, technical reports, and

workplans that specify the timing of

training and expansion of services

May be outdated and may

not reflect current policies,

strategies or context.

Demographic • Demographic and Health Survey,

Reproductive Health Survey, national

census data, Population Reference

Bureau data

• Data on population growth and trends

• Data on population characteristics,

e.g., geographical distribution, age,

gender, occupation

• Behavioral surveillance surveys

Data needs to be adjusted

from the survey year to the

present and projected time

period.

Data may not reflect the same

time period and, therefore,

cannot be easily aligned.

Data specific to use of NUMS is

usually nonexistent or limited.

Services • HMIS reports, program M&E reports,

facility surveys of service records, daily

registers

• Reported number of family planning

services provided

• Number of providers trained and

facilities equipped to offer the method

Particularly for NUMs,

data may be unavailable,

outdated, incomplete, or

unreliable for the past 12

months. Plans for training

providers, generating

demand, etc., also should be

considered.

Research studies • Any research available on pilot

programs, operations research, and

scale-up studies on a particular method

Smaller scale studies,

especially pilot studies, are

often very controlled scenarios

that do not necessarily reflect

the reality of introducing a

method into programs.

Family planning

program experiences

• Key informant interviews with program

managers, implementing partners, and

technical experts

• Any reports/briefs available on program

experiences with the same product or a

previous generation of the product

• RHInterchange, an online database

that documents contraceptive orders

by year/country

• Social marketing sales figures

May lack quantitative inputs

or data provided may be

unreliable.

RHInterchange only captures

data on orders placed, not

actual consumption.

New methods may not be

included.

Program targets • National policy and strategic planning

documents

• National annual program targets or

service coverage rates set as goals for

the program

Program targets may be

politically motivated for

advocacy purposes and not

based on realistic program

capacity or likely real

demand.

*Adapted from Table 6-3: Types and Sources of Data for Forecasting Product Consumption (USAID | DELIVER PROJECT

2011a)

Section

Forecasting for NUMs

Build out assumptions based on a contextual framework

of factors that will potentially influence the uptake of

NUMs

At the crux of assumption-based forecasting are key contextual demand factors

that determine method uptake. These factors are shaped by various inputs,

which are the result of the current social, political, and economic influences

in the country. They also include the success and/or limitations of the program

that supports the introduction and/or expansion of the NUM. Our research

has identified four primary contextual factors that must be considered in the

forecasting methodology for NUMs:

1. CLIENT,

2. PROVIDER,

3. FINANCE, and

4. AVAILABILITY

These contextual factors are inter-related and should be considered inclusively

when preparing a forecast. Table 3 outlines these factors, the inputs and

influences that shape them, including examples of secondary data sources

that can support assumption building around these factors. Note that the larger

political environment—including policy and government commitment—has a

cross-cutting impact on the way each factor actually plays out.

Table 3: Key Contextual Demand Factors for NUMs

Inputs Influences

Example

Data Sources

POLICY & GOVERNMENT COMMITMENT

CLIENT

Awareness

Knowledge

Attitudes

Need

Program inputs, such

as IEC campaigns and

promotions

Current method mix

Cultural factors

DHS

RHS

LMIS (logistics data)

Behavioral surveillance

studies

Qualitative research

PROVIDER

Knowledge

Capacity

Perceptions

Training

Supervision

Standard treatment

guidelines

RHS

Qualitative research

FINANCE

Cost to client

Intermediate costs

(cost to program)

Finance for

procurement

Comparative costs

Cost/affordability of

product

Willingness/ability of

donors, governments,

and others to procure

product

Inclusion of product in

insurance scheme

DHS and RHS

National population

census data budget

Program finance data

Procurement budgets

Willingness-to-pay studies

Donor, government, and

other policies regarding

product

AVAILABILITY

# of outlets (facility/

community-based;

public/private sector)

Geographic scope

Supply constraints

Supply chain capacity

Strength of reporting

system

Complimentary

products

Expiration rates

RHS

Supply chain assessments

Stockout rates for existing

methods

Service provision

assessment

Logistics indicator

assessment tool

Forecasting for NUMs

Section

1. CLIENT

Consider the consumer perspective. What cultural norms and beliefs are likely to

inhibit/facilitate client acceptance of the method? What behavioral and social

network patterns support projections in method uptake? Consider awareness,

attitudes, knowledge, practices, beliefs/myths, gender dynamics, need,

preferences, and supply constraints as inputs.

Key health and demographic characteristics can provide information about the

client perspective: (1) contraceptive prevalence by method and contraceptive

method mix, (2) incidence of unplanned pregnancies, (3) unmet need for family

planning, (4) current total fertility rate (TFR) and TFR target, (5) current family size

and ideal family size, (6) maternal mortality ratio, (7) women of reproductive age

seeking abortion services, and (8) population growth. There may also be research

on issues that include the role of men in method choice and use, support in the

community/family for family planning methods, and many other topics that could

affect demand for and uptake of particular NUMs.

Data sources to support client-based assumptions include DHS, RHS, research

reports, health information and management system (HMIS) reports, service data,

qualitative research, and behavioral surveillance studies.

Assumption-building tips:

 Is there a target audience for the NUM in question? Divide the population

into family planning user groups based on the method mix available in

that country. Fully understand the potential user group for the NUM. Some

guiding questions include:

• What percentage of women will adopt a new method? What are the

perceived benefits of the method? Who is likely to adopt this method

when it is first introduced? Who is likely to adopt it later? In similar

programs/countries, what was the adoption rate?

• What cultural norms and beliefs inhibit/facilitate client acceptance of the

method?

• What impact will the new method have on existing methods? Will the

new method be adopted only by new clients? If so, then past trends for

existing products may continue at the same rate. If not, estimate how

many women using other methods may switch to the new method.

• What are the specific needs and preferences of women with respect to

family planning? Consider geographic and cultural differences. Would

one method be more appealing to women in some geographic and/or

cultural regions than in others?

• Where would clients access the NUM, and would they have preferences

about where to get their family planning method (i.e., from a community

health worker, at a health facility, or in the private sector)?

• If the client is expected to pay for the NUM, will they be ready and

willing to do so? How does the cost of using this NUM over time compare

to the cost of using the existing methods?

• Was there a pilot study completed for the NUM in the country, or

another similar country? What did the findings suggest about the

acceptability of the method to clients? Would any characteristics

of the study group make acceptance more or less likely among

the population as a whole? If using pilot data, it is very important to

note any major changes in the pilot design compared to the current

program—for example, there could be major differences if one project

relies on the private sector while the other works through public sector

outlets. Additionally, there could also be major differences in outcomes

if program inputs—information, education, and communication (IEC)

campaigns—are significantly different.

 IEC campaigns, social marketing strategies, or any other promotional

activities for family planning methods are designed to increase usage

by making clients more aware of the availability and benefits of a

product and/or service. These campaigns have the potential to influence

awareness/knowledge/preferences/attitudes of clients. Programs need

to consider the planned promotional activities for the NUM and how

effective these campaigns will be to motivate method uptake. Usually

uptake increases during a concerted campaign, although it may fall if

the campaign is not sustained. If there are no awareness-building and

demand-creation activities planned, the method uptake will likely be very

low. With NUMs in mind, consider these questions:

• What is the estimated percentage of increased consumption following

a promotional activity? Will interest decrease after the campaign? Will

clients continue to use the method? (Consider whether or not a user

must return to a service delivery point frequently to obtain the method.)

Will new clients continue to adopt it?

Forecasting for NUMs

Section

• How strong are social networks in the country, and how fast will positive

(or negative) information about a new product permeate a network? If

satisfied/dissatisfied clients have a tendency to talk to others about their

experience, how will this impact the rate of method uptake?

 Increasing CPR and reducing unmet need are common goals of family

planning programs. For forecasts, the challenge is determining the realistic

rate of increase for CPR from one year to the next, looking at the methods

already available; which new methods will be introduced and/or scaled;

and what are the anticipated growth rates for each method. Each year,

especially when new methods are introduced, some clients switch to a

more appropriate method for them or become a family planning user for

the first time. Questions to consider:

• What is the projected rate of growth in CPR? Will it be a steady increase

during each of the next five years, or will there be a slow increase at

first, and then a burst of adoption in later years? Or, will there be a rapid

increase at the start of the period, which will taper off as the program

progresses? The assumption on rate of increase of CPR will have a

significant impact on the forecast.

• Is the projected growth in CPR realistic? This depends on the maturity

of the program and the interventions that are implemented to increase

CPR.

• What has been the previous growth rate for other NUMs in the country?

 Demographic data can provide information about overall unmet need.

However, two assumptions need to be made: (1) what percentage of

women of reproductive age will start using this particular method when it is

available?; and (2) how quickly will the method become available, based

on the service delivery strategy? If there are supply constraints (e.g., if

facilities need to order stocks but are not trained to do so, or if the forms for

ordering commodities from a central supply unit do not include the NUM),

the method will not be readily available and uptake will be affected. (See

more under “4. AVAILABILITY”).

2. PROVIDER

The uptake of specific family planning methods is directly linked with provider

knowledge, attitudes, and behaviors. If providers are not adequately trained

and do not understand the new method, they will emphasize it less when

counseling clients. They may share misinformation about the method that, in turn,

may perpetuate misconceptions about the NUM. If offering the NUM is perceived

as a burden for over-worked staff, providers may not offer the method regularly.

Program managers need to assess if the method requires a highly trained

provider (e.g., to insert an IUD), or if the method can be offered by varying levels

of providers, including community health workers (e.g., female condoms and

CycleBeads). The speed at which providers can be trained will also influence

how rapidly the method will be offered and used. To determine if providers are

able to inhibit/facilitate uptake of a NUM, consider the following inputs and

influences for assumption building.

Assumption-building tips:

 What are the standard treatment guidelines (STGs) in the country and other

policy guidelines that influence provider behavior? Are these guidelines

widely known and followed? Is the NUM included?

 Where will the NUM be offered (public vs. private sector, community vs.

facility level, etc.)? Consider what skills are needed to offer the NUM and

which levels of providers (e.g., public vs. private sector, community vs.

facility level, etc.) will offer or currently offer the method.

 How many providers are trained or will be trained to offer the method?

• What is the training plan for preparing providers to offer the NUM? Given

the training plan, how rapidly can services be rolled out to new sites?

• How long does it take for varying levels of providers to be comfortable

and competent offering the method?

• What is the plan for coordinating supervisory visits and/or conducting

refresher trainings to reinforce knowledge?

 What issues/challenges may providers have with offering the method?

• How has provider bias influenced family planning programs offering this

method (or similar methods) in the past, or in other similar contexts? Was

a pilot study completed for the NUM in the country, or another similar

country? What did the findings suggest about provider attitudes and

acceptability?

Forecasting for NUMs

Section

• What is the time required for counseling on the method and how does

that compare to other methods currently offered?

• Are there any surgical procedures required to offer the method? What

additional resources are required to offer the method and what are the

stock levels of those supplies? What level of buffer would be necessary

to account for human error (e.g., when these resources are damaged in

use, etc.)?

3. FINANCE

Many financial issues need to be considered when preparing a NUM forecast.

There may be inhibiting/facilitating factors associated with commodity costs,

particularly the program’s ability to purchase the anticipated required quantity

of the product; the clients’ ability to pay for the product, when relevant; and the

health facilities’ ability to offer the product (e.g., is special equipment needed,

are there incentive schemes in place, such as performance-based financing,

that affect service delivery, etc.)?

Willingness and ability of donors and governments to procure the product

will depend on product cost, budgets; and policies related to such factors as

product registration, emphasis on certain types of products (e.g., long-acting

and permanent methods [LAPM], injectable contraceptives), and preferred

manufacturers (PATH 2009).

Programs have to examine the costs to clients (e.g., any repeat costs for

continued use, willingness-to-pay through public/private sector outlets when

relevant), intermediate costs to the facility/health delivery system/distributor,

and comparative costs of the method to other existing methods. These costs

can be influenced by the larger health system; e.g., if the public health system

provides the method free of charge or not, if the method costs are covered by

insurance, and/or if the product is available through both public- and private-

sector channels. Secondary data sources with respect to finance factors include

the DHS, RHS, and national population census data.

Forpublichealthprograms,availabilityofnancingforcommodityprocurementshouldnotbeused

to constrain a forecast. A forecast is the estimated future demand assuming full availability of supply.

Funding constraints can be factored into roll-out plans, supply plans, and scale up after the forecast is

complete.

Assumption-building tips:

 Public-sector issues:

• Can the program afford to procure enough of a product to meet

demand? Are donors or other sources willing and able to procure the

product?

• Are performance-based financing structures in place, and, if so, is the

NUM included in the system? Is there an incentive/disincentive to offer

the product and how will this affect the health facilities’ willingness to

offer the product?

• Will clients be expected to pay for the product in the public sector? Can

clients afford to pay for the method at the price point set?

• Has funding for promotional strategies for family planning been

allocated? How far has planning and implementation progressed? Is the

NUM included? If not, is there an opportunity to do so?

 Private-sector issues:

• Is there a business case for distributors/pharmacies/social marketing

organizations for offering the product? How much of the product will

they need to sell to make a profit?

• Can clients afford to pay for the method at the price point set?

• What is the competition for the product in the market? Is there more

than one brand of the product, and how does this affect price and

demand?

4. AVAILABILITY

Another influential uptake determinant is the availability of the commodity. How

long it will take to get the product in the country, whether it will be available

through both public- and private-sector outlets, whether it will be available

country-wide or just in specific areas, and how rapidly it can be moved from a

central location to these sites all will affect the number of the commodity that

should be procured. The forecast evaluation around this factor will consider

market conditions; in-country procurement, registration, and import regulations;

and current national family planning policies and norms. It is essential that

those responsible for conducting the forecast have a strong understanding of

the program design and implementation plan that will support the introduction

Forecasting for NUMs

Section

or scale-up of the NUM, including the service capacity, the service delivery

strategy, and awareness-building campaigns. Additionally, availability depends

directly on the supply chain capacity in the country, including the effectiveness

of the inventory management, storage, and distribution systems, and

functionality of the logistics management information system (LMIS). This analysis

needs to cover both public- and private-sector plans, when relevant. Data from

the LMIS and service provision assessment (SPA) can inform answers to these

questions; other data sources will need to be considered.

Assumption-building tips:

Consider public- and private-sector implications for each question.

 How long will it take to get the commodity in to the country, and how does

this time frame impact the forecast? Think about how long it will take from

forecasting, placing the order, manufacturing the product (if needed),

shipping the product, getting the shipment cleared from customs, and

getting the product to the distribution center/warehouse.

 What is the reach of the program (e.g., national, district, etc.) and what is

the number of service delivery points (SDPs) that can offer the NUM within

the country? How will this number change over time?

 Will it be feasible to incorporate the NUM into the existing supply chain

in a timely manner (i.e., can the NUM be easily integrated into storage

facilities, reporting forms, distribution processes)? As such, will the supply

chain be able to get the NUM to appropriate SDPs? How long will this take?

Is the country’s pipeline short enough to deliver the NUM to SDPs before it

expires?

 What is the distribution strategy for getting the product to SDPs? Consider

both the initial supply and re-supply.

• Initial supply: How is initial stock distributed in the country? For example,

do you give a small quantity to all facilities and then re-supply based on

need?; do you distribute based on the population, and give more to the

highly populated areas?; or do you give more to the larger facilities and

less to the smaller?

• Re-supply: How is re-supply handled? If the facilities need to order stock,

does the order form currently include the NUM? If not, how will the order

be placed? If yes, do facilities know how to order re-supply?

 What activities are planned to generate demand? When will these

activities be implemented? Is there a risk that awareness generation

activities will out-pace the availability of the supply?

 Are there reporting and monitoring systems in place to gather information

on product usage patterns, and is the NUM included?

• If not, when will the NUM be included? What other mechanisms will be

in place in the meantime to gather information on product uptake and

usage?

• Are providers trained and competent to record new method users? Are

new users being documented accurately?

• If the NUM is included in reporting and monitoring systems, what is the

expected lag time for the data to be collected and utilized (monthly,

quarterly, every six months?) Who receives and analyzes the data, and is

the data shared with all stakeholders?

 Are there additional “complementary” products that accompany the

NUM? For NUMs, such as IUDs and implants, consider additional pieces of

equipment for insertion/removal, and how the availability of these supplies

affect provision of the method.

 Consider the process and timeline for re-ordering supplies to replenish

the stock in-country. There may be more flexibility in the private sector to

re-order supplies when they are needed, but public sector procurement is

usually done annually. What implications does this have for the forecast?

Host an assumption-building workshop with key

stakeholders

Given the special case of forecasting for NUMs, it is recommended that a

workshop be held with various key stakeholders, including those in the public

and private sectors, if relevant, to lay out the forecasting assumptions for

the particular NUM. Bring to the table information gathered from steps 1–2.

Determine the key factors that are expected to influence demand/uptake

of the method as a group. Agree on how the forecast will be calculated and

document all assumptions that will be made to yield the forecast.

Forecasting for NUMs

Section

Forecast! And, run a “reality check” on the quantification

and distribution strategy

The next step is to actually do the forecast. For guidance on how to build a

forecasting model that is appropriate for the needs of the program, refer to the

reference manuals and tools available in Section 7: Resources.

After the demand forecast is calculated, check assumptions and calculations

and ensure that others are involved in reviewing the quantification. It is very

important to build in a “reality check” into the process and ask if the forecast

seems to be logical.

At this point, it is also critical to think through what will happen after the supply

enters the country. Ask questions like how and where will it be stored; how will

the supply reach SDPs; do people need to be trained on how to store/distribute

the new product; and how will method use be reported, etc.? Will facilities be

able to order the product? Specifically for countries that use maximum-minimum

inventory control systems

, how will facilities know how much of the NUM to

order the first time because there is no maximum established? Also, how will

facilities restock the NUM if it is not yet integrated into the ordering system? Utilize

on-the-ground experts in supply chain management to assess if the distribution,

reporting, and restocking plans seem realistic.

Develop and implement a monitoring plan

It is very important that programs understand that forecasting is a dynamic

process—especially for NUMs—and monitoring is necessary to evaluate programs

and indicate if corrections are needed. Continuous monitoring is critical when

introducing and/or scaling a NUM because it is difficult to predict how uptake

will actually occur. Monitoring will allow programs to identify any systemic issues

that are inhibiting uptake, as well as to understand product usage patterns

(e.g., analyze if there is a difference in uptake in the public/private sector,

if applicable; analyze where the product is/is not popular and why, etc.).

Monitoring will help programs keep track of stock imbalances, particularly when

“A max-min inventory control system is designed to ensure that the quantities in stock fall within an

established range. The max stock level is the level of stock above which inventory levels should not

rise, under normal conditions. The min stock level is the level of stock at which actions to replenish

inventory should occur under normal conditions. Most successful inventory control systems used for

managing health commodities are max-min systems of one type or another” (USAID | DELIVER PROJ-

ECT 2011a).

there are sudden increases in consumption and more stock needs to be ordered

to meet demand.

Initially, monitoring may be challenging because NUMs may not be included

on commodity ordering or tracking forms, monitoring and supervision tools, or in

service data collection forms. A short-term solution to consider is to implement

“spot checks” to collect both quantitative and qualitative data on how the

introduction and/or scale-up of the NUM is going. For the longer term, programs

should advocate for the NUM to be added to the monitoring and evaluation

tools and surveys used in the public sector. If it is being offered in the private

sector, sales data will support monitoring and evaluation efforts. To the extent

possible, design the tools used in the short term to emulate the long term

perspective.

Forecasting for NUMs

Section

Common Pitfalls

Now that we have reviewed the recommended steps for approaching how

to forecast for NUMs and reviewed how to build assumptions to support

forecasting, it is important to highlight a number of common pitfalls that occur

when forecasting for NUMs. Consider the following when you approach any

forecasting exercise for NUMs:

 The stakes can be high when introducing or scaling up a NUM in a country.

Often, these efforts are supported by large donor investments and may

involve multiple global and local organizations. If the introduction or

scale-up of method availability does not lead to significant uptake of

the method, within a defined project period, donors may decide not to

continue providing support for the method. Donors and programs need

to set realistic goals for client uptake and consumption rates. When stakes

are high, programs often project a higher estimated uptake than actually

occurs, with serious consequences of over-supply, wasted resources, and

unmet program goals. Financing for contraceptives is often a zero sum

game—over-forecasting of one method usually means fewer resources and

potential stockouts of others. To the extent possible, programs should seek

the expertise of an unbiased third party to support forecasting the demand

for NUMs. This would ensure that program aspirations do not interfere with

realistic forecasts.

 On the other hand, financing constraints may limit how much of a particular

NUM can be procured. If procurement of contraceptives is determined

nationally, it is possible that NUMs will not be assigned appropriate

forecasting numbers due to budgetary limits. Countries may be reluctant

to spend scarce resources on methods whose appeal to potential clients is

unknown. A way to address this issue is to ensure that programs that work

with NUMs have a seat at the decision-making table during the national

quantification and forecasting process.

 Over-reliance on issues data in lieu of consumption data is dangerous

when forecasting for underused methods. Some programs depend on issues

data—which is based on the movement of products between any two

storage facilities within a country (e.g., when the regional level distributes

Section

Common Pitfalls

supplies to the district level)—as a proxy for consumption data.

Weak

supply chains may be unable to provide reliable or timely consumption

data. This is a problem because contraceptives may be distributed in a

country in anticipation of demand, but the demand may not materialize.

Thus, issues data does not point to actual consumption and use. Programs

that can consider issues data in their forecast need to account for this

discrepancy and reduce their forecast amounts accordingly by (1)

forecasting conservatively, because it can be assumed that products

issued are not necessarily consumed; and/or (2) schedule small initial

shipments of the product into the country until monitoring data is available

to confirm or challenge forecast assumptions.

 Needs-based forecasts can estimate unrealistically high quantities. Needs-

based forecasting does not require historical program data. Instead, it

depends on inputs from demographic or behavioral surveillance surveys.

Such a tactic establishes that if, for example, one million people report

having an unmet need for family planning, that all one million people will

access family planning, without considering availability, access, and/or

the various socio-cultural barriers that may exist. This forecasting exercise

tends to yield unrealistically high forecasts because it (1) overestimates the

actual demand, and (2) does not consider if the product is also available

in the private sector. Programs that rely on needs-based forecasts need to

account for the discrepancy of inflated forecasts and reduce their forecast

amounts accordingly.

 Note differences in forecasting for public- and private-sector family

planning programs. There are inherent market differences between the

public and private sectors. Whereas a method might be very popular in the

private sector—where women can access the method privately, over the

counter, and at their convenience, for instance—this may not be possible in

the public sector. Forecasting should not be based on the assumption that

a “private sector” experience will produce the same results as a “public

sector” experience, and vice versa.

 Often forecasts occur only once a year and do not allow for course

corrections. Demand forecasts, especially if compiled nationally for the

public sector, usually only occur once a year. This becomes an issue as

Consumption data provide information about the quantity of goods actually given to or used by

customers (USAID | DELIVER PROJECT 2011a).

the base assumptions shift during the year and course corrections cannot

be made. This is especially true for NUMs because uptake may change

significantly as a method is introduced or scaled up—programs are limited

in being able to predict uptake, and are further limited when they cannot

make course corrections and order more of a product mid-cycle. Those

who procure contraceptive commodities need to be aware of this issue

and build mechanisms for course corrections into the procurement process,

such as instituting pipeline monitoring and regularly revisiting the supply

plan.

 Regulations, product approvals, and essential medicine lists could

inhibit programs from even getting the product into the country. Some

countries adhere to strict regulatory mandates that medical supplies

need to be approved or registered by the government before they can

enter the country. Others only allow products listed on the World Health

Organization’s (WHO) Essential Medicines List to pass customs (or to pass

customs without prohibitive fees). Getting approval for a new commodity

can be a long, bureaucratic process. Programs must plan for proper

approvals and buy-in before moving forward with procurements of NUMs.

Once in-country, the method may be subject to quality control testing even

if it is tested by the supplier before shipping. The accuracy and reliability of

in-country testing may vary greatly, and can result in long clearance delays

or even quarantine. It is important to know the testing regime, and whether

or not the government will accept pre-shipment testing or must perform its

own tests after arrival. The caution for programs and others who forecast for

NUMs is that forecasts need to be timed appropriately, according to these

restrictions and regulations.

Common Pitfalls

Section

Tips from the Experts

Key informants shared important advice and experiences. Their insights support

the assumption-building process and provide method-specific examples.

“

Remember that there are always nuances to the data available per the

method you are forecasting. For EC, we used demographic data (in Liberia,

Rwanda and Benin) because there was a question of quality of services and

consumption data. We also looked at statistics on violence against women

when forecasting for EC, assuming that there is a sexual component of

violence. For IUDs and implants, there can be issues of infections due to high

humidity and therefore client acceptance. For implants, there was also an

issue of provider training on removal.—JSI, Liberia/Rwanda/Benin

“

Variability in forecasted demand versus actual demand can be caused by

funding challenges, overly ambitions projections, and changes in program

mandates. Also, delays in the implementation of training programs assumed

by service providers and the re-assignment of skilled staff to other duties led

to missed consumption targets.—JSI, Ghana

“

For the Sino-implant (II), a new contraceptive, it took a lot longer than

expected to register the product in many countries. This delay had

implications on the accuracy of our 5-year projection.—FHI 360, Global

“

You can use the previous generation of a product to give some indication

of demand for the next generation. The projections for Sino-implant (II) in

Indonesia were based on the large number of Norplant users.—Population

Council, Global

“

The availability of skilled providers should be taken into account. If you do

not have skilled IUD providers, you don’t get uptake when the product is

introduced. While this is the same for implants, providers can be trained

much more quickly to insert implants.—Population Council, Global

Section

Tips from the Experts

“

Beware of overly optimistic target setting at least as a basis for procurement

decisions. Your forecast should be based on what women will reasonably

use and not what policy-makers want them to use or think they SHOULD

use. Just because a family planning method is safe, effective, easy to use

that does not mean women will choose to use it. Establish various scenarios

for a forecast in order to put boundaries on what the forecast will be—for

example an optimistic, realistic and pessimistic scenario.—JSI, Global

“

Even with trained providers, motivated clients, and a supply of CycleBeads

in the central warehouse, the fact that CycleBeads were not integrated into

the MIS meant that consumption was not being tracked. The logistics staff

at the regional and health center level did not have an easy way to order

CycleBeads, unlike methods already included on the supply requisition form.

—IRH, Global

“

Over-ordering EC causes problems, as with many other products, because

there is an expiration date. This can also have large policy implications and

reflect poorly on the MOH that approved the order.—ICEC, Global

“

With EC, there is a keen wish for privacy and anonymity—it is much more

private to go to the pharmacy. There is also a speed issue—women like

being able to get EC over the counter at a pharmacy rather than relying

on public sector clinic hours from 9-11. Consider women’s preferences

in forecasting, such as what are the easy access points?—ICEC, Kenya

example

“

When you’re forecasting for a NUM, you don’t have to set an ultimate target

if you can establish a good relationship with a supplier. Once you have that

relationship, you can re-order supply if needed. Think about negotiating

payment terms, such as paying up front rather than paying after 30 days, if

that moves the product faster.—DKT International, Ethiopia (social marketing

perspective)

“

Implants have been around in Kenya for a long time, but they are

underutilized. The quantification this year is based on consumption trend

data, which generally increases year over year, and program inputs

(awareness raising, etc.). We added 2-3% to the forecast to account for

increase in demand. Uptake for this method is very slow, so we know not to

over exaggerate.—Division of Reproductive Health, MOH, Kenya

“

With CycleBeads, it has been very normal for early uptake to be rather

minimal because this method is very different from what people are used

to. Demand increases over time as more people in the community have

experiences with it and adopt it and as providers get used to offering it. So

what initially may seem like an over-supply may actually be a good quantity.

However, while CycleBeads do not have an expiration date, consider that

storage costs do have to be accounted for.—IRH, Global

Submit your own “Tips from the Experts” regarding your NUMs forecasting

experiences and lessons learned here at http://tinyurl.com/Submit-a-Tip

Tips from the Experts

Section

Taking the Discussion

Forward

By helping countries and programs maximize forecasting accuracy, particularly

for contraceptives that are not accounted for in existing procurement tools,

this forecasting guide helps address three key objectives: (1) strengthen existing

procurement systems, (2) enhance the use of resources, and (3) increase the

knowledge base. Addressing this procurement challenge ultimately has the

potential to (1) increase access to contraceptive commodities and (2) expand

contraceptive choice at the country level.

Moreover, our goal for this guide is that it will influence how forecasting for NUMs

will occur in the future—and that this is a step in the right direction. By utilizing

a set of defined factors to inform assumption making as part of the forecasting

process, programs can base decisions on a standardized framework that will

present a realistic picture of future markets for NUMs. We are calling for a

systematic approach to forecasting for NUMs. However, we cannot reach that

goal without documenting, monitoring, and accessing forecasts and forecast

performances for NUMs over time. This will take commitment from programs and

individuals worldwide.

We ask that you commit to taking the discussion forward. Here’s how you can

contribute:

1. Submit your own “Tips from the Experts” regarding your NUMs forecasting

experiences and lessons learned to http://tinyurl.com/Submit-a-Tip.

2. Participate in generating awareness around NUMs forecasting and

increasing the knowledge base by engaging in online discussions and

forums around the topic. When possible, track and share program data to

build out data sources for NUMs forecasting.

3. Suggest recommendations and feedback for the guide—especially if you

use it to support a forecast—on the guide’s K4Health toolkit here.

Section

Taking the Discussion Forward

Resources

Logistics and Procurement Guides

 The Logistics Handbook: A Practical Guide for the Supply Chain

Management of Health Commodities, USAID | DELIVER PROJECT, 2011a

The Logistics Handbook explains the major aspects of logistics

management, with an emphasis on contraceptive supplies. It is intended

to help managers who work with supplies every day, as well as managers

who assess and design logistics systems for entire programs. In addition,

policymakers, system stakeholders, and anyone working in logistics will also

find it helpful as a system overview and overall approach.

Key terms and concepts are clearly defined and explained; the document

includes detailed information about the design and implementation of

logistics management information systems and inventory control systems.

Overviews of quantification, procurement processes, as well as storage,

transport, and product selection, are also included.

 Quantification of Health Commodities: Contraceptive Companion Guide,

USAID | DELIVER PROJECT, 2011b

This guide should be used to support the forecasting step in conducting a

quantification for contraceptive supplies, following the project’s approach

to quantification. The guide presents a methodology for forecasting

consumption of contraceptives and the additional supplies needed to

provide both short-acting methods (SAM) and long-acting and permanent

methods (LAPM) of contraception. Throughout the guide, examples of

forecasting for a SAM (contraceptive pills) and a LAPM of contraception

(introduction of contraceptive implants) are presented to illustrate the data

sources, forecasting assumptions, and the outputs at each step.

 Quantification of Health Commodities: A Guide to Forecasting and Supply

Planning for Procurement, USAID | DELIVER PROJECT, 2009

This guide for quantification of health commodities was developed to assist

technical advisors, program managers, warehouse managers, procurement

officers, and service providers in (1) estimating the total commodity needs

and costs for successfully implementing national health program strategies

and goals, (2) identifying the funding needs and gaps for procuring the

Section

Resources

required commodities, and (3) planning procurements and shipment

delivery schedules to ensure a sustained and effective supply of health

commodities.

 Procurement Capacity Toolkit, Tools and Resources for Procurement of

Reproductive Health Supplies, Version 2, PATH, 2009

As more developing countries take on responsibility for purchasing health

commodities, requisite capabilities—such as decision-making, planning,

and technical skills—often require strengthening. To address this need, PATH

developed the comprehensive Procurement Capacity Toolkit for those

responsible for and involved in the supply of reproductive health products.

Particularly relevant for forecasting is Procurement Capacity Toolkit, Version

2: Module 1: Defining Reproductive Health Supply Requirements

 Contraceptive Security: Ready Lessons II. Lesson 8. Expanding

Contraceptive Choice through Support for Underutilized Methods,

Commodities Security and Logistics Division, Office of Population and

Reproductive Health, Bureau for Global Health, 2008

Ready Lessons I and II are two series of booklets that provide USAID Missions

and their partners strategies and activities that can significantly improve

contraceptive security. Ready Lessons I introduces the fundamentals of

programming for contraceptive security. Ready Lessons II shows how to

apply these basics in the context of a rapidly changing health environment,

including changes in development assistance, health sector reforms, and

growth of the private sector. The section highlighted above focuses on

special considerations related to ensuring adequate supply and distribution

of NUMs.

Forecasting Tools

 PipeLine, USAID | DELIVER PROJECT

PipeLine is a best-in-class desktop software tool that helps program

managers plan optimal procurement and delivery schedules for health

commodities; it also monitors their orders. Policymakers, product suppliers,

and donors can generate reports, estimate future product needs, and

use the software as a key tool in program planning. This effective tool has

been used in more than 40 countries around the world, with products

in reproductive health, essential medicines, anti-retroviral testing and

treatment, malaria testing and treatment, lab supplies, and tuberculosis

treatment. Note that PipeLine is not a forecasting tool, although it can

help organize historical data and estimate forecasts. Its real value is as a

quantification and supply planning tool. PipeLine can take a future forecast

and, using stock on hand data, minimum and maximum stock levels, lead

times etc., estimate how much of a commodity will be needed and when.

 REALITY √, EngenderHealth, ACQUIRE Project, 2007

Reality√isafamilyplanningprojectiontoolwithastraightforwardExcel

Workbook that allows the user to assess past trends in the contraceptive

prevalence rate (CPR) and test future scenarios for the geographic area

where the program is operating. The tool also allows users to test and assess

whether established goals are reasonable, based on the local, specific

context.Reality√wasdesignedformultipleaudiencestorunprojections

based on the geographic focus of their choosing. Beneficiaries of the tool

would include MOH planners and administrators at the national, provincial,

or district levels; as well as family planning programmers at donor agencies

or cooperating agencies. Anyone with basic Excel skills will be able to

use the tool. The tool was also designed to be a stand-alone product

that could be used in low-resource settings, where high-capacity Internet

connections or high-level programming skills may not be available.

 Module F: Regulation, Procurement, and Distribution of a Progestin-Only

ECP in Resources for Emergency Contraceptive Pill Programming: A Toolkit,

PATH, 2004

This toolkit facilitates the integration of the emergency contraceptive pill

(ECP) into developing country family planning and reproductive health

programs. It includes resources for ECP advocacy, assessment, service

provision, and evaluation. The planning and implementation tools represent

best practices and experience that will help programs move through the

steps required to make ECP services routinely available through health

service delivery systems. The toolkit shares an array of materials developed

by PATH and by other organizations when they worked in a variety of

settings to incorporate ECPs into family planning services. By bringing

together these resources in a format that facilitates their use, the toolkit can

reduce duplication of efforts, redundancy, and unnecessary expense.

Resources

Section

 CycleBeads® Procurement Toolkit, IRH, PATH, and JSI, 2010

This document presents program managers with a model to forecast

an initial supply of CycleBeads in their country or region over a five-year

period. CycleBeads supports women who want to use the Standard

Days Method® of family planning. The document provides instructions for

completing an Excel worksheet that calculates an estimated CycleBeads

order based on potential demand. The last page of this document is an

example of a completed worksheet.

 Spectrum, Health Policy Initiative

Spectrum is a suite of policy models that uses of unified set of Windows-

based commands, which can be easily learned. The models project

the need for family planning and reproductive health, maternal health,

and HIV/AIDS services. Most models are available in English, French, and

Spanish. Some are also available in Portuguese, Arabic, and Russian. Each

model includes a detailed user manual that not only describes how to use

the software but includes sections on data sources, interpretation and use

of the results, a tutorial, and a description of the methodology.

Databases

 The RHInterchange, Reproductive Health Supplies Coalition

The RHInterchange provides access to up-to-date, harmonized data on

more than U.S. $1 billion worth of shipments of contraceptive supplies

for more than 140 countries around the world. The RHInterchange stores

historical information and offers visibility into upcoming shipments. You

can use it for pipeline monitoring, commodity management, analysis, and

planning.

 Supplies Information Database (SID), Reproductive Health Supplies Coalition

The Supplies Information Database (SID) is an online reference library

with over 6,000 records on the status of reproductive health supplies at

the country level. The library includes studies, assessments, and other

publications dating back to 1986, many no longer available even in their

country of origin. SID’s user-friendly search and feedback features allow

you to locate, download, and print materials. Documents can be searched

by keyword, country, date, subject area, and even type of publication.

Continuously updated, SID offers you exclusive access to the latest supply

information on more than 230 countries and territories worldwide.

Reports and Briefs

 Technical Briefs on New and Underused Reproductive Health Technologies,

The Caucus on New and Underused Reproductive Health Technologies,

2012

These peer-reviewed briefs provide concise, but comprehensive,

technical overviews for 13 underused reproductive health technologies,

including contraceptive implants, CycleBeads, diaphragm, emergency

contraceptive pills, female condom, HPV vaccines, levonorgestrel

intrauterine system, magnesium sulfate, manual vacuum aspiration, medical

abortion, misoprostol for maternal health, oxytocin, and progesterone

vaginal ring. Each brief includes information on efficacy, suppliers, pricing

agreements, and more. The Caucus on New and Underused Reproductive

Health Technologies is a community of practice that was established under

the Reproductive Health Supplies Coalition; PATH is the Secretariat.

 Emergency Contraceptive Pills: Supply Chain Considerations, USAID |

DELIVER Project, 2012

Emergency contraception is an important component of reproductive

health programs. To ensure the routine availability of emergency

contraceptive pills, the managers of public health supply chains consider

the unique characteristics of this important method.

 Frequently Asked Questions: Caucus on New and Underused Reproductive

Health Technologies, Reproductive Health Supplies Coalition, 2011

This document answers common questions raised by members of the

Reproductive Health Supplies Coalition and its Working Groups about

the Caucus on New and Underused Reproductive Health Technologies,

its goals, organization, and functions. The document includes a vetted

definition for new and underused reproductive health technologies.

 A Risky Business: Saving Money and Improving Global Health Through

Better Demand Forecasts, Center for Global Development, Global Health

Forecasting Working Group, 2007

Great strides have been made in the last decade to improve health in

poor countries—more aid funding for drugs and vaccines; creation of

funds to buy medicines; and concessionary pricing of medicines by some

pharmaceutical firms. However, the global supply chain that connects

Resources

Section

the dots—production to people—does not work well. The problem is

poor forecasting of effective demand for products. Good forecasting is

fundamental for key decisions, such as how much production capacity

to build, which must be made years in advance of products being

delivered. But, donors that provide much of the money to purchase

drugs, and a whole range of technical agencies and intermediaries,

have yet to devise and coordinate among themselves or with developing

country governments, credible forecasts. This report of the Global Health

Forecasting Working Group provides an analysis of the problem and a

sensible agenda for action. The report offers specific recommendations

that apply across a range of products and that could be implemented by

identifiable public and private organizations.

 Accurately Forecasting Contraceptive Need: Levels, Trends, and

Determinants, USAID | DELIVER PROJECT, 2007

Information on the expected accuracy of the contraceptive forecasting

processes is useful for family planning supply chain managers to

efficiently plan and procure contraceptive commodities and to maintain

uninterrupted supplies to meet clients’ needs. This study examines the

accuracy of the contraceptive forecasting processes of 81 family planning

programs in 30 developing countries, using time-series records between

1994 and 2005 for past contraceptive consumption and projected needs.

Forecast accuracy is defined as the absolute percentage difference

between the actual and projected quantity of a contraceptive dispensed.

An analysis of 1,586 one-year-ahead contraceptive forecasts indicates

that the expected median absolute percent error for one-year-ahead

contraceptive forecasts for public sector family planning programs is

about 25 percent. Multiple regression analysis indicates that the forecast

accuracy of public sector programs has been improving over time, which

is partly attributable to an improved family planning logistics management

information system performance and the use of forecasting software.

Interview List by Organization

Organization Type of New and/or

Underused Method

iPlus Solutions Female condom

ICEC Emergency contraception

DKT/Ethiopia Emergency contraception

Population Council/Kenya Emergency contraception

MOH/DRH/Kenya Various

JHPIEGO/Kenya Various

MSH/Kenya Various

IRH/Guatemala CycleBeads

IRH/Rwanda CycleBeads

FHI Sino implant

Population Council Implants, IUDs

PSI/Nepal IUD

JSI Various

Appendix 1

Appendices

Discussion of Variability in MAPE

What could be the reasons for the observed variability in NUMs

versus non-NUMs forecast error rates?

Median Absolute Percent Error (MAPE) in Forecasts by Method Type

2008 2009 2010 Average

New and underused methods (NUMs) 50% 145% 36% 77%

Other methods (male condoms,

oral pills, injectables)

22% 17% 32% 23.7%

Difference between NUMs and other methods:

53.5%

Based on the data provided, we see a lot of variability in the overall median

error rate for NUMs for the three years of data we have. By contrast, non-NUMs

show significantly less variability in forecast error. What could be the causes of

this variability?

1. First, it is important to emphasize that these error rates represent the median

error of a variety of different methods and countries, over time. Between

2008 and 2010, more countries’ data were increasingly available to

analyze, and more countries added new methods to their forecasts. The

addition of the initial forecast error rates for countries with data that is newly

available, or that added new methods, could cause some of the variability

in the MAPE.

2. The NUMs guide suggests that the lack of historical data makes it difficult

to forecast for NUMs, and result in higher forecast error rates and higher

forecast error variability than non-NUMs. Indeed, in the countries analyzed,

the longer a country has been forecasting for a particular product, the less

variable its error rates tend to be. However, the correlation between error

variability and length of forecasting history is weak.

Table 1 shows the forecast error rates for IUDs from 2006 to 2010 in eight

countries, some with as much as five years’ forecast history. Negative numbers

reflect instances where the forecast was higher than the actuals reported;

positive numbers show where the forecast was lower than the actuals. Some

countries, despite a longer history of forecasting, have large errors and a great

difference in error year to year.

Appendix 2

Forecast Error Rate

for IUDs, Country

2006 2007 2008 2009 2010

Ghana -33% -33% -10% -42% -24%

Liberia N/A N/A N/A 18% 76%

Malawi N/A N/A -204% -100% 66%

Mozambique -245% 0% -250% 38% 36%

Paraguay -21% -65% -36% -35% -21%

Rwanda 15% 15% -153% -8% 16%

Tanzania -67% 3% -4% 11% 21%

Zambia N/A N/A 72% 52% -172%

Median -33% 0% -36% 2% 19%

Table 2 shows the forecast error rates for implants from 2006 to 2010 in five

countries. Interestingly, Malawi appears to have a greatly increased error rate

despite having more years of data; this is possibly related to the impact of

missing data for 2007.

Forecast Error Rate

for Implants, Country

2006 2007 2008 2009 2010

Ghana -65% -91% -36% 10% -119%

Malawi -43% N/A -8% 30% -780%

Rwanda 76% -82% -25% -57% -34%

Tanzania 21% 17% 37% -256% 6%

Zambia N/A N/A 82% -13% -94%

Median -11% -82% -8% -13% -94%

As the data for Mozambique (IUDs) and Malawi (implants) show, variability in

forecast error is not explained solely by the length of forecast history. In addition

to poor quality data and inaccurate forecast assumptions, other factors may

contribute to variability in forecast error:

 Registration processes and delay approving registration for new methods

and products (this particularly impacts new IUDs, implants, and emergency

contraceptive pills).

 Lack of funding commitment for commodities or supportive training

programs (for example, training in implant insertion), donor fall-through, or

general financing delays.

 Manufacturing problems causing delay in product availability.

 History of prolonged product stockouts or low stocks (especially if a product

Appendices

Table 1

Table 2

is totally or somewhat fungible with another product already available): this

makes it difficult to predict client behavior when the product is available.

Clients may have switched to another method during the stockouts, and

only some smaller portion of those clients may again be interested in their

previous method when it becomes available.

 While stockouts are excluded from forecast error calculations, they

nonetheless affect uptake once the product is in-country.

Data from the 2006–2010 USAID | DELIVER

PROJECT’s Procurement Planning and Monitoring

Report (PPMR) and PipeLine

Appendix 3

Appendices

Link:

Appendix 3_PPMR Data 2006-2010_NUMs Forecasting Guide.xls

Reproductive Health Supplies Coalition

Rue Marie-Thérèse, 21 · 1000 Brussels, Belgium

Tel: +32 (0)2 210.02.22

Fax: +32 (0)2 219.33.63

www.rhsupplies.org

communications@rhsupplies.org

Access the K4Health Toolkit at

http://www.k4health.org/toolkits/NUMs-forecasting-guide