Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes?

Estimates of the modern contraceptive prevalence rate (mCPR), a population-level indicator, that are derived directly from family planning service statistics lack sufficient accuracy to serve as stand-alone substitutes for survey-based estimates. However, data on contraceptive commodities distributed to clients, family planning service visits, and current users tend to track trends in mCPR fairly accurately and, when combined with survey data in new tools, can be used to approximate the annual mCPR in the absence of annual surveys.


Long Acting and Permanent Methods Long Acting and Permanent Methods
Tubal Ligation (  Couple-years of protection (CYP) are designed to capture the impact of a given contraceptive method on providing protection against unintended pregnancy. While this measure can be very useful for program monitoring, it poses a challenge for estimating contraceptive coverage or users, specifically for long-acting methods.

Short-Term Methods Short-Term Methods
For all modern methods, the CYP factor is applied to commodities and visits data in order to calculate the number of years of coverage provided. For short-acting methods, this coverage is fully realized in the year the services are provided. Therefore, total CYP in a year for shortacting methods can be used as a proxy for the number of women using each method at midyear. For long-acting methods, the CYP factor instead represents the number of years each individual receiving a method is protected into the future.
In a steady state, the CYP for long-acting methods can be used as a rough estimate of users. For example, it would assume that for each implant inserted, there are 2 to 3 women still using implants inserted in previous years. However, using the CYP can overestimate users during a period of rapid scale-up and underestimate users during a period of decline, as the entire impact of the method is counted in the year it is distributed, rather than spread over the years it is in use (see Figure A3.1)

Figure A3.1: Illustration of CYP for Long-Acting Contraceptives in 3 Scenarios
A second issue that arises when estimating users of long-acting methods is that while discontinuation of short-acting methods would generally be captured through service statistics (in that short-acting users are required to return regularly to receive their method, so stopping would result in fewer commodities and services being distributed), discontinuation of longacting methods cannot be tracked based on commodity insertion or visit data alone.
In order to address these issues, a different approach was used to calculate "estimated users" of long-acting and reversible contraceptive (LARC) methods. The methodology used here is consistent with the methodology used by Marie Stopes International's Impact 2 model. 17 Calculating the number of users of a LARC, such as intrauterine devices (IUDs) and implants, in a given year requires combining the following: 1. An estimate of the number of women provided with the method in that year (based on commodities/services data). 2. An estimate of the number of women who are still using a LARC provided in previous years.

Calculating users provided with a method in the given year (item 1 above):
first-year discontinuation rates (see Table A3.1) are applied to the commodity/service data for LARCs (IUDs and implants). These are applied to the figures that have been adjusted to account for the missing private sector data as described in the body of this paper.

. Method-Specific Continuation Rates
Calculating users still using a method provided in a previous year (item 2 above): For this calculation, the number of LARC insertions in past years is needed. Some of this data is available within a health management information system (HMIS), but depending on how long an HMIS has been in place, a full trend of historic data may not be available. In this case, historic services are estimated based on available data. This is done by projecting back a linear trend from the oldest year of data available; the trend is projected backward based on the duration of the given method (10 or 5 years for IUDs and 5, 4, or 3 years for implants). These estimates are needed to calculate the total users in each year; for example, IUD users in 2010 can include women who received their IUD as long ago as 2000. An illustration of this process is shown in Once a full trend of service provision data is available, the cumulative number of users still using a method received in past years can then be calculated. This is done by applying the method-specific continuation rates shown previously in Table A3.1. Essentially, a cohort of users is traced out for each set of insertions in past years, then, by summing across these cohorts, the total number of users still using a LARC received in past years can be calculated. Adding in the number of women using a LARC based on those who received a method in the current year (item 1 from above), the total number of users can be estimated. An illustration of this is shown in Table A3.3, where it is estimated that 16,255 women are using an IUD in 2009.

Table A3.3. Estimating Total Number of Users From Trends in IUD Insertions
This methodology allows for more accurate estimates of the total number of LARC users, especially in cases where there is rapid scale-up or declines in the number of insertions. However, this methodology is not without limitations. Projecting a simple linear trend backward based on method duration provides a means of estimating some level of historic use, but it does not fully represent historic use. This methodology may underestimate continuing users in countries with a long history of method provision or relatively consistent levels of provision in years prior to available data. Alternative approaches are being explored that can better account for estimating historic trends. However, even with these limitations, we feel that this approach is still preferable and more accurate than simply using the CYP factor for LARCs.
Finally, it should be noted that we did not apply this methodology to sterilization numbers, as such a calculation is more complex and because in most countries, overall sterilization numbers are fairly low and/or relatively static (in which case the CYP factor is a decent approximation). However, there are a few countries where the use of the CYP factor for sterilizations does not provide an accurate estimate, and work is underway to develop an approach comparable to that described for LARCs to apply in these cases.

SUPPLEMENT 4. Formulae for Computing the Components of Mean Square Error
The calculations of the components of mean square error (MSE) were performed as described below.
Variance was measured as the squared deviations between the annual estimated modern use (EMU) rate and the value for the corresponding year on the EMU least-squares linear trend line; the sum of these was divided by n, in which n = the number of observations. That is, VAREMU = ∑(EMUyear i -EMU Trend Lineyear i) 2 / n Where: n = the number of years for which service statistics-based estimates are available.
Note that the formula we used to calculate variance differs from more commonly used versions in that it divides by the number of years rather than by the mean EMU. The formula we used gives the average of the squared deviations across the years, rather than the single mean for all the original observations. The use of the former is more appropriate when dealing with time series data.
Next, because countries with higher modern contraceptive prevalence rates (mCPRs) have a potential for greater variability in EMU rates than countries with lower mCPRs do, we used the relative variance (RelVar), which adjusts variance according to the overall level (mean) of EMU.
The RelVar was calculated as:

RelVarEMU = VAREMU / MeanEMU
Where: MeanEMU = the average of EMU values included in the variance calculation. As with estimates of variance, the level bias estimates were adjusted to account for the fact that countries with higher mCPRs had potential for higher levels of bias measured in absolute terms. This potential distortion was mitigated by dividing the estimates of absolute level bias by the mean value of the FPET-based mCPR estimates during the reference period for each country (as shown in the formula above).
Trend bias was measured as the absolute value of the difference between the 2 slopes: 1 for the service statistics-based EMU rates and 1 for the survey-based FPET estimates of the mCPR. That is,