Case-Study Evaluation

CDC/HRSA Maternal and Child Health

Epidemiology Program (MCHEP)

1990 - 1996

Prepared by

University of Illinois at Chicago

School of Public Health

Evaluation Team

Stacie Geller, PhD

Arden Handler, DrPH

Joan Kennelly, RN, MPH

For

Centers for Disease Control and Prevention

Division of Reproductive Health

Pregnancy and Infant Health Branch

March, 1998

Final Summary and Findings

Case-Study Evaluation

CDC/HRSA MCHEP

1990 - 1996

Funded in part by Centers for Disease Control and Prevention through a cooperative agreement #S279-15/15 from the Association of Schools of Public Health.

This evaluation report could not have been accomplished without the assistance of the many individuals in the state health agencies of Arizona, California, Florida, Georgia, Maryland,

Oregon, South Carolina, Washington, Washington DC, as well as within CDC and HRSA who were willing to spend their time arranging and participating in site visits and conference calls, reviewing materials, and taking the time after the site visits to clarify areas of confusion.

This report was also dependent on the assistance of two staff persons at the University of Illinois School of Public Health. Melissa Rubin who arranged all of the site visits, prepared all materials and ensured that the site visits went smoothly, and Dana Pinker who assisted in the final production of the report.

For more information, please contact:

Arden Handler, DrPH

Community Health Sciences

School of Public Health

University of Illinois at Chicago

2035 West Taylor

Chicago, Illinois 60612

(312) 996 - 5954

handler@uic.edu

Executive Summary…………………………………………………………………......i

I. National Background and History…………………………………………………….....1

II. Overview of Study Design………………………………………………………………..2

III. Selection Criteria for Comparison States…………………………………......………..4

IV. Components of the Study Design…………………………………………………….....4

V. Instrument Development……………………………………………………………..…..6

VI. Site Visit Protocol……………………………………………………………………..….7

VII. Overall Observations on Effectiveness and Value of the

CDC/HRSA MCH Epidemiologic Capacity in State Health Agencies………....……..8

VIII. Contextual Factors Promoting and Inhibiting MCH

Epidemiologic Capacity in State Health Agencies……………………………..……...11

IX. Conclusion…………………………………………………………………………….......25

X. References……………………………………………………………………………......26

XI. Appendix A: Benchmarks of Effective State MCH Epidemiology…………………....27

In 1987, the Centers for Disease Control and Prevention (CDC) Pregnancy and Infant Health Branch within the Division of Reproductive Health and the Maternal and Child Health Bureau of the Health Resources and Services Administration established the Maternal and Child Health Epidemiology Program (MCHEP) to enhance the capacity of state MCH programs to engage in effective public health surveillance and data-based decision-making. In May 1996, the University of Illinois School of Public Health (UIC-SPH) was the recipient of a cooperative agreement between the Association of Schools of Public Health/CDC to evaluate MCHEP.

The MCHEP evaluation employed a case-study approach to assess participating states’ epidemiologic capacity, through both an historic and contemporary examination of each state’s MCH epidemiology activities as well as an examination of relevant contextual factors and exogenous variables. The purpose of the evaluation was twofold: to provide evaluative information to participating states for the ongoing implementation, improvement and management of their respective MCHEP analytic and epidemiologic efforts; and, to identify and characterize factors which promote or inhibit the institutionalization of state-based MCH epidemiology.

The evaluation design included four MCHEP states (California, Georgia, Washington, Washington, D.C.), four comparison states (Arizona, Florida, Maryland, Oregon) and one pilot state (South Carolina). The UIC-SPH evaluation team conducted site visits to each state to assess the state’s MCH epidemiologic efforts and activities and to meet with key informants and stakeholders to understand the context in which MCH epidemiologic efforts have developed over time (between 1990 and 1996). The evaluation relied on a number of tools, including a questionnaire to assess each state’s status on benchmarks of epidemiologic capacity in four functional areas: Vision and Planning, Infrastructure, Analysis and Utilization, and Translation and Dissemination. The benchmarks were developed based on domains suggested by CDC’s Chronic Disease Surveillance Branch.

Observations from the nine states, suggest that the CDC/HRSA Maternal Child Health Epidemiology Program is an effective and innovative strategy for enhancing MCH analytic and epidemiologic capacity. In general, states with MCHEP as compared to states without MCHEP have both the necessary analytic leadership and focus to allow them to more effectively engage in data-based decision-making throughout the MCH planning cycle.

Findings from this evaluation also indicate that the effectiveness of MCHEP varies greatly based on key factors operative in each of the states. These include the following: the vision of the state and its commitment to data-based decision-making, the level of infrastructure and state fiscal and personnel support, the placement of the assignee in the state’s organizational structure, the relationship of the assignee to the MCH/Title V program, the assignee’s prior training and background, relationships with local health agencies, the extent of collaboration with partners outside the health agency, source of funding for the assignment, and the longevity of the assignment.

As the CDC/HRSA MCHEP program appears to be a very effective strategy for increasing epidemiologic and analytic capacity for MCH within state health agencies, this program deserves the strong financial and verbal support of the leadership of CDC and HRSA. MCHEP should be viewed as one vital arm of a comprehensive strategy to increase the ability of state MCH programs to carry out the core functions of public health. It is time to support the expansion of MCHEP so that all states have the necessary analytic leadership enabling them to more effectively plan and evaluate programs, as well as develop policies to improve the health status of women, children and families.

Since the late 1980’s, in response to the Institute of Medicine's (IOM) landmark report, The Future of Public Health,¹ and the data related mandates established for the Maternal and Child Health Block Grant through the Omnibus Budget Reconciliation Act of 1989 (OBRA 1989), state Maternal and Child Health (MCH) programs have increasingly become interested and involved in improving their capacity for effective public health surveillance and data-based decision-making.

In 1987 the Centers for Disease Control and Prevention (CDC) Pregnancy and Infant Health Branch within the Division of Reproductive Health and the Maternal and Child Health Bureau (MCHB) of the Health Resources and Services Administration (HRSA) established the Maternal and Child Health Epidemiology Program (MCHEP) to enhance the capacity of state MCH programs to engage in effective public health surveillance and data-based decision-making. During the first 10 years of the program, epidemiologists were assigned to state health agencies to support the development and enhancement of MCH epidemiologic capacity (recently, MCHEP as a CDC-only initiative has also been providing resources to states without the assignment of personnel). MCHEP assignees are jointly supported by CDC and the state's Title V Block Grant funds (or other funding sources such as the Preventive Health and Health Services Block Grant); however, assignees are CDC employees.

In 1991, the Pregnancy and Infant Health Branch of CDC's Division of Reproductive Health announced a package of cooperative agreements, furthering its support for state-based epidemiology. Cooperative agreements were awarded for the following three programs: Prenatal Smoking Cessation (PSC), Pregnancy Risk Assessment and Monitoring (PRAMS), and Centers for Healthy Infants and Pregnancies Surveillance (CHIPS). Applications for CHIPS required application proposals for PSC and PRAMS as well. The CHIPS program was designed to enable states to build analytic capacity through the development of multi-disciplinary teams.

While remarkably different in their approaches and the amount of federal resources invested, the purposes of both MCHEP and CHIPS were fundamentally comparable. That is, both programs were designed to build MCH epidemiologic capacity, to develop comprehensive MCH surveillance, to translate data into information and engage in its dissemination, to provide technical support for state and local health departments, and to conduct relevant research projects.

Although participation in CDC/HRSA's MCHEP has steadily increased since the inception of the program, with demand usually outweighing resources, an evaluation of the ability of these programs to expand state level MCH epidemiologic capacity had never been undertaken. In October 1995, an investigator-initiated proposal to conduct an assessment of the various state efforts for enhancing MCH epidemiologic capacity through MCHEP and CHIPS was successfully submitted to CDC. In May, 1996 a cooperative agreement between the Association of Schools of Public Health (ASPH)/CDC and the University of Illinois at Chicago (UIC) School of Public Health (SPH) was established to support implementation of the MCHEP/CHIPS evaluation.

As a result of CDC's decision to discontinue funding of their CHIPS cooperative agreements, the evaluators were asked to focus their efforts on the four currently funded MCHEP states. It was also agreed that one state which had participated in both MCHEP and CHIPS, would serve as the pilot state.

The MCHEP evaluation was designed to assess the MCH epidemiologic and surveillance capacity of MCHEP states, through an historic and contemporary examination of each state's MCH epidemiology program as well as through an examination of relevant contextual factors and exogenous variables. The participating MCHEP states were: Washington, Georgia, California, and Washington, DC. The purpose of the evaluation was twofold: i) to provide evaluative information to participating states for the ongoing implementation, improvement, and management of their respective analytic and epidemiologic efforts; and ii) to identify and characterize factors which promote or inhibit the institutionalization of state-based MCH epidemiology. Identification of promoting and inhibiting factors was expected to advance the effective implementation of MCHEP efforts in other states and locales.

The MCHEP evaluation used a participatory case-study design. The case-study method was selected as it is well suited for the assessment of projects which have been implemented in a variety of uncontrolled settings and changing environments ^2-4.The strength of this design is that it is empirically based, utilizing multiple data sources apropos to the evaluation's information needs. While the case-study approach attempts to capture with as much reliability as possible the relationships among the myriad factors operating within each state's MCH program and the larger socio-political system, this approach makes no assumptions about comparability across state programs.

The participatory nature of the study design required significant efforts on the part of those states involved, promoting maximum participation from a wide variety of vested interests as either key informantsⁱ or stakeholdersⁱⁱ. It was anticipated that responding to the concerns of both key informants and stakeholders would broaden and strengthen the evaluation process, enhance the quality and relevance of the findings, and increase the utility of the evaluation report ^5-6.

Nine states participated in the evaluation. The design included one pilot (South Carolina), four MCHEP, and four comparison states. Comparison states were included in the evaluation in an attempt to account for the development of enhanced MCH epidemiologic capacity given the broader technologic, economic, and political changes which have taken place across the nation over the past decade.

Comparison states were chosen based on the following two criteria: (1) Location in the same geographic region as an MCHEP state; and (2) Past participation in the Enhanced Analytic Skills Program (EASP) of the Maternal and Child Health Training Program (MCHTP) at the University of Illinois - School of Public Health, which was assumed to reflect interest in increasing MCH epidemiologic capacity in the state. EASP was a continuing education program designed to enhance the analytic skills of state-level public health professionals involved in the collection and utilization of MCH data. The comparison states selected were Oregon, Florida, Arizona, and Maryland.

There were four components to the evaluation strategy. Components I and II involved only the MCHEP states and together were designed to accurately portray the characteristics of each state’s MCH epidemiology program. These included the underlying premises of the distinct programs, their respective goals, objectives, and activities, and the micro-environment within which these took place. Components III and IV involved both MCHEP and comparison states and were designed to assess elements of the broader macro-environment influencing the development of MCH epidemiologic capacity within states. A detailed description of each component follows.

Component I assessed implementation of MCHEP based on responses to a self-administered questionnaire. It examined: i) the degree to which specific program objectives were accomplished; ii) the degree to which the program was implemented overall; iii) the nature of state and other resources required to successfully accomplish program objectives; and, iv) the extent to which program implementation led to the further development and institutionalization of MCH epidemiologic capacity and analytic activities.

Component II captured qualitatively, the meaning of the responses given in Component I. Through key informant interviews, the context for the development and implementation of each state's MCH Epidemiology Program was assessed. Component II examined whether the specific combination of resources, activities and administrative arrangements that shaped each program, facilitated or hindered achievement of its objectives.

Component III attempted to measure the development and institutionalization of effective MCH epidemiology through state’s self-ratings on benchmarks of MCH epidemiologic capacity in four functional areas: Vision and Planning, Infrastructure, Analysis and Utilization, and Translation and Dissemination (Appendix A). The benchmarks were developed based on domains suggested by CDC’s Chronic Disease Surveillance Branch. They are both empirically and research based, and reflect some of the state-of-the-art thinking on the scope and extent of effective state MCH epidemiologic activities^7-10.

The benchmarks provided profiles of states' MCH epidemiologic efforts and activities for two points in time. Time one (T1), the baseline, was the year 1990, and time two (T2) was the year 1996. Enactment of the federal Omnibus Budget Reconciliation Act of 1989 (OBRA '89) mandated uniform reporting requirements for Title V programs leading state MCH programs to begin to more systematically wrestle with issues involving standardized data collection, analysis, and utilization to improve the health status of the MCH population. Therefore, we expected to see varying increases in epidemiology capacity across states after 1990.

Component IV captured qualitatively, the context and meaning of the responses given in Component III. Through key informant and stakeholder interviews, the relevant social, political, and economic circumstances within each state which have the potential to impact MCH analytic capacity, as well as the organizational structure of the health agency and its MCH and epidemiologic units, were explored. Additionally, stakeholders' perceptions of the state's system of MCH epidemiology and surveillance were assessed.

The questionnaire for Component I was based on each assignee’s initial workplan which outlined specific goals and activities to be achieved in the MCHEP assignment. The workplans were developed, in most cases, in coordination with the assignee, the MCH Director, and a CDC supervisor. For questionnaire development, workplans were reduced to both general and specific activities. Where workplans were not available, specific goals and activities were gleaned from the assignee’s annual reports. The extent to which goals and objectives had been achieved was assessed using a categorical scale ranging from "not at all" to "a lot". A separate questionnaire was developed for the pilot state’s CHIPS program, using their 1991 CHIPS CDC Cooperative Agreement Proposal.

Component III

Benchmarks within each of the four functional areas delineated above were developed by UIC staff in cooperation with representatives from MCHEP and CHIPS states (Appendix A). The benchmarks were piloted in South Carolina and appropriately revised. Benchmarks were translated into yes/no questions, and for select benchmarks, the extent to which the benchmark was achieved, was also captured.

The benchmarks were designed for use in conjunction with key informant and stakeholder interviews. They are meant to provide an assessment of each state’s capacity at a particular point in time and are not designed for use in cross-state comparisons.

The participatory nature of the study design as described above required significant effort on the part of each state to promote maximum participation from a wide variety of vested interests. The CDC/HRSA assignee or MCH designee in comparison states was asked to identify key informants and stakeholders to participate in site visit interviews. They were also asked to identify an evaluation workgroup (usually some or all of the key informants) to complete the questionnaires (Components I and III) that were the basis of the site visit interviews.

Prior to each site visit, state’s were provided with a summary description of the evaluation's purpose and design and asked to distribute this to key informants and stakeholders. Also, a month prior to the site visit, Component I and Component III questionnaires were completed by each state’s evaluation work group. These were then returned to the UIC evaluation team for review. Questionnaire responses from Components I and III assisted in the development of site visit interview questions for Components II and IV.

One week prior to the site visit, UIC mailed copies of the completed questionnaires to the key informants and stakeholders with a reminder of the date and time of their interviews. These individuals were asked to review the documents with the expectation that during the site visit they would provide feedback about the completed questionnaires and offer concrete, descriptive examples to support and elaborate questionnaire responses.

The UIC evaluation team conducted each site visit over a 2-3 day time period. Key informants were generally interviewed as a group, while stakeholders were generally interviewed individually. In most cases, MCHEP state site visits began with an interview with the assignee and/or the MCH Director, while comparison state visits began with the MCH Director and/or an MCH program designee. Whenever possible, a closing session was also held for these same individuals. In all cases, the UIC evaluation team made themselves available for follow-up communication as needed.

This section of the evaluation report provides insights as to whether and how the presence of an MCH epidemiologist assigned by CDC/HRSA to a state health agency enhances state MCH analytic capacity.

The work of the state-based MCH epidemiologist is not typical of most epidemiologists in state health agencies. MCH epidemiologists use the tools and the population-based focus of epidemiology to enable the state health agency to effectively carry out the activities of the MCH planning cycle which include surveillance, assessment, program planning, program monitoring, evaluation, and policy development. As such, the MCH epidemiologist provides analytic leadership to enhance the ability of MCH programs to carry out the core functions of public health. Specifically, the MCH epidemiologists assigned to the CDC/HRSA MCHEP were charged with the following:

1. Apply epidemiologic analysis to maternal and child health services issues such as service utilization, access and effectiveness.
2. Analyze data for programmatic planning and priority setting within state MCH programs.
3. Develop state capacity to implement data collection systems needed by the state to assess MCH population needs. Perform state-specific studies and projects.
4. Further develop a state’s capability to evaluate the effectiveness of existing programs in maternal and child health.
5. Serve as a resource for the implementation and the evaluation of innovative pilot and intervention programs and/or studies to improve the health of mothers and children.
6. Assist the state to identify and classify existing gaps in analytic skills and to create and implement long-range plans to meet them.

Observations from the nine states included in this evaluation, suggest that the CDC/HRSA Maternal Child Health Epidemiology Program is an effective and innovative strategy for enhancing MCH analytic and epidemiologic capacity. In general, states with MCHEP as compared to states without MCHEP have both the necessary analytic leadership and focus to allow them to more effectively engage in data-based decision-making throughout the MCH planning cycle. States with an MCHEP program are also part of a larger CDC network which supports their analytic efforts; this support serves as an additional stimulus for state MCH analytic activities.

Findings from this evaluation also indicate that the effectiveness of MCHEP varies greatly based on key factors operative in each of the states. These include the following: the vision of the state and its commitment to data-based decision-making, the level of infrastructure and state fiscal and personnel support, the placement of the assignee in the state’s organizational structure, the relationship of the assignee to the MCH/Title V program, the assignee’s prior training and background, relationships with local health agencies, the extent of collaboration with partners outside the health agency, source of funding for the assignment, and the longevity of the assignment. (These factors will be discussed in depth in the next section).

States in which MCHEP has been in place for an extended period of time, in which the epidemiologist has been successfully integrated into the workings of the state health agency, and in which the assignees’ efforts are driven by the priorities of the MCH program are more likely to be able to translate a state’s commitment to data-based decision-making into enhanced analytic efforts aimed at MCH program planning and policy development. In these states, there is an increased likelihood that advanced analytic techniques will be applied to data generated from surveillance systems to plan and evaluate programs as well as develop policy.

There appears to be a great value in having a highly skilled leader present in state health agencies to coalesce analytic activities and promote the use of data for decision-making on behalf of the MCH population. In the majority of the comparison states visited, the single factor most inhibiting the further development of analytic capacity was the lack of a designated MCH epidemiologist. There was a tremendous amount of willingness, knowledge and skill among MCH staff in these state health agencies and in fact, the majority have been able to build some analytic capacity in their MCH programs; however, in none of these states was there one person responsible for guiding and coordinating MCH analytic efforts.

Because MCHEP offers state MCH programs the opportunity to have a highly skilled epidemiologist (a CDC/HRSA assignee) temporarily join their staff, this program addresses the major need found in the comparison states: an individual to provide leadership to guide and coordinate MCH analytic efforts. Without such leadership, analytic efforts often do not take place at a level which enables a state to make a decision based on data, or are not part of an overall coordinated plan for data-based decision-making.

Given the success of MCHEP, it seems essential that this program continue to be a high funding priority for both CDC and HRSA. In addition, both agencies should seek increased funds to be able to expand such efforts to as many states as possible. With the many changes occurring among state health agencies around the country, ongoing assessment of the implementation of MCHEP should be routinely conducted by CDC/HRSA to support and strengthen its effectiveness.

This section provides a discussion of the contextual factors identified through the Maternal and Child Health Epidemiology Program (MCHEP) evaluation which appear to have influenced the ability of states to generate, analyze, and use data in the formulation and evaluation of sound programs and policies intended to improve the health and well-being of women and children. This discussion summarizes observations from all nine states visited during the evaluation and is presented in the context of defining what is important for an effective MCH epidemiology program supported by CDC/HRSA. This section uses a broad definition of MCHEP and includes both an MCHEP assignee or dedicated activities engaged in by states to support expanding MCH analytic capacity. Broadening the definition of MCHEP enables us to provide both funders and states with information about the factors that promote effective state MCH epidemiology which can be applied beyond the strategy of an assignee. In some instances, however, specific factors are discussed only in terms of an assignee.

Before discussing the contextual factors which promote effective MCH epidemiology, there are several underlying assumptions about the components of an effective MCH epidemiology effort at the state level which must be mentioned. The goal of MCHEP is to improve the use of data for decision-making on behalf of the MCH population and therefore, the role of the state MCH epidemiologist is to facilitate the state’s ability to carry out the activities of the MCH planning cycle which include surveillance, assessment, program planning, program monitoring, evaluation, and policy development. Therefore, the majority of the MCHEP effort should be driven by the MCH needs and priorities of the state rather than by the interests or needs of CDC, HRSA, or the individuals designated to work as part of MCHEP.

In addition, although epidemiologists are typically trained in etiologic epidemiology (i.e., the etiology of adverse health outcomes) the work of the MCH epidemiologist is applied epidemiology (i.e., the use of epidemiology to carry out the activities of the planning cycle). Etiologic epidemiology is only one tool used by the MCH epidemiologist as he/she participates in analytic activities throughout the planning cycle. Analytic studies and etiologic research are necessary to understand the relationship between risk factors and health status outcomes; however, an effective MCHEP must translate study findings into information that can be used to target high risk populations, develop programs, launch interventions and implement policies.

What follows is a listing and description of the factors associated with effective state MCH epidemiology:

A. Organizational Structure of State Health Agency. In order for MCHEP to be successful, the organizational structure of the state health agency should be considered before the placement of a CDC/HRSA assignee or before MCHEP activities are launched. While organizational structures differ across state health agencies, whatever the structure, MCHEP must be strategically placed so that the MCH epidemiologic activities can build on and promote the existing epidemiologic capacity within the state agency and make that capacity readily available and accessible to MCH programs. Factors related to state health agency organizational structure which should be considered before the initiation of MCHEP include:

1. Organizational arrangement of MCH program/role and responsibility of the Title V/MCH Block Grant director. State Maternal and Child Health programs are structured differently. Of importance is whether the Title V/MCH Block Grant Director and staff are seen as the leadership for MCH activities within the agency, whether or not Title V provides the funding for these activities. When the designated MCH director, typically the Title V/MCH Block Grant director, is not sufficiently empowered and/or does not have adequate collaborative relationships or partnerships to make and influence decisions regarding programs and policies that affect the entire MCH population, the efforts of MCHEP may not be sufficiently broad to allow for maximal impact on information-based decision-making to improve the health of women and children. To facilitate MCHEP’s effectiveness, the unit within the state health agency which provides leadership and establishes priorities for MCH must take a broad view of MCH, understanding that there are a plethora of programs in addition to Title V which should be included as part of the MCH bailiwick.

2. Location and function of the epidemiology unit. The epidemiology unit is an essential partner to MCHEP; therefore the placement of the epidemiology unit within the health department, the MCHEP relationship to the unit, and the general approach and focus of the unit are all critical factors in the establishment of an effective MCHEP. In particular, the epidemiology unit must appreciate the importance of applied epidemiology as an appropriate role for epidemiologists in a state health agency.

Understanding the previous relationship of the epidemiology unit to the MCH program is essential; units which have had no prior relationship with the MCH program are much less likely to understand or be supportive of the activities of the MCHEP. In particular, whether they have previously collaborated with the MCH program for planning and evaluation activities provides a partial indication of whether the epidemiology unit has only an etiologic orientation or whether they are also involved in applied epidemiology. Prior involvement by the epidemiology unit in applied epidemiology suggests that there will also be support for this kind of effort through MCHEP.

3. Relationship between the MCH program and designated data units within and external to the state health agency. An effective MCHEP requires easy and direct access to quality data. Consequently, essential to an effective MCHEP is the relationship between the MCH program and the other units in the state health agency as well as throughout state government that are responsible for data collection, data analysis, data reporting, and data systems development and management. In addition, supportive and collaborative working relationships with other epidemiologists and data units in the state health agency are likely to provide technical and collegial support to the MCHEP. If these relationships have not been previously developed by the MCH program, with the initiation of an MCHEP, it is incumbent that efforts be made to ensure that these relationships are supported and encouraged.

4. Strength of MCH program within the State Health Agency. The influence of MCH within the state’s public health decision-making processes contributes to the effectiveness of MCHEP. While the role of MCHEP is to enable an MCH program to become a leader in setting MCH policy, it is clear that when MCH has a history of being strong and visible to the political leadership in the state, MCHEP receives more support to carry out its responsibilities and is therefore, better able to turn data into information for program planning and policy development.

In sum, strategic placement of MCHEP is necessary. Two possible approaches that appear likely to maximize the effectiveness of MCHEP are placement within the same organizational structure as the MCH program, or within an epidemiology unit with direct ties to the MCH program. Wherever MCHEP is located, its activities and efforts should be guided by the state’s MCH needs and priorities rather than by the interests or needs of CDC, HRSA, or the individuals designated to work as part of MCHEP.

B. State Support for Information-Based Decision-Making is Key. MCHEP is most likely to succeed in states in which there is commitment at multiple levels for the collection, analysis, and reporting of quality data to support information-based decision making to improve the health of women and children. Emphasis on a population-based approach to the activities of the public health agency is essential, with those at the helm dedicated to the use of data to plan programs and make policies. In cases in which the director of the state health agency does not have a public health orientation, at a minimum, there needs to be a commitment to information based decision-making at the level of the state MCH director, typically the Title V/MCH Block Grant director.

C. State’s Assessment of its Analytic Capacity. Critical to the success of MCHEP is the state’s own assessment of their MCH epidemiologic capacity. Using the benchmarks developed through this project (or a similar tool), a state should develop a baseline profile of its epidemiologic capacity. Based on this assessment, a long-range strategic plan for institutionalizing capacity should be developed with MCH priorities clearly delineated. The state can then monitor progress made towards enhancing and institutionalizing their epidemiologic capacity, assessing improvements over time.

D. Infrastructure Support. Adequate infrastructure support is essential for successful implementation of MCHEP. MCHEP must have the resources necessary to carry out its tasks, and therefore must have a role in shaping the state’s strategic plan related to resources for enhancing the state’s MCH analytic capacity. This strategic plan must consider the effective use of existing staff resources as well as plans for seeking additional resources in order to provide sufficient analytic and administrative support for the state’s growing analytic activities.

Whether an MCHEP assignee has their "own shop" (e.g., CA, SC, GA) with analytic staff working directly for him/her, or is more of a "freelance" consultant (e.g., DC and WA) in which he/she works without their own staff, primarily making use of analytic capacity found throughout the agency, an MCHEP assignee (and likewise, an MCHEP program not based on an assignee) needs access to adequate administrative, analytic, and technical support in order to accomplish the myriad activities necessary for effective MCH epidemiology. The components of infrastructure which can maximize the effectiveness of MCHEP include:

1. Computer and analytic staff. A sufficient number of adequately trained personnel to support data collection, computer programming, and analysis, whether or not these individuals work directly for MCHEP, is necessary. Adequate state funds and sufficient state FTE lines to support hiring, training, and retention of these staff should be sought.

2. Capacity of MCH program staff. The analytic skills and capabilities of the MCH program staff, and whether they appreciate the importance of using data to guide their activities are important factors affecting MCH analytic capacity within a state. Even if they are not in possession of the analytic skills themselves, these individuals need to be able to interact effectively with those who have these skills. To ensure this, training aimed at effectively communicating data needs should be provided on a continual basis. While some of this training should be provided by MCHEP, states should also make use of the various training opportunities provided through HRSA and CDC.

3. Hardware and software. To insure that MCHEP can effectively carry out its planned activities, adequate hardware (this includes "user friendly" mainframes as well as personal computers) and software needed for data collection and analysis should be supported by state investment. However, if state support is not forthcoming, such capacity (particularly for PCs) should be funded through external financial resources (e.g., grant dollars and special projects funds).

(a) Management information system. To the extent that state health agency program planning and policy development is dependent on the collection and analysis of data from its programs, data systems that are able to collect and generate timely, high quality data from these programs are essential. States’ investments in improving the quality of their Management Information Systems will clearly facilitate the use by MCHEP of data for program planning, monitoring and evaluation.

(b) Integrated information system. While most states have a management information system, few of these systems allow for the integration of data across programs, and even fewer allow for easy integration of program data with population-based datasets. Many states however, are in the process of developing such systems or improving outdated systems. States should be encouraged and supported to develop data systems using common definitions and unique identifiers so that MCH and related program data (e.g., family planning, WIC, etc.) can be easily linked to each other and to datasets such as vital records or Medicaid. The development and maintenance of an integrated information system, which includes both population-based data as well as program data will increase the ability and opportunity of MCH epidemiologists and others involved in MCH epidemiologic activity to effectively use data for program planning and policy development. If a vision for such a system exists, MCHEP is likely to become a vital partner in bringing this vision to fruition.

5. Other financial support. Funding beyond that provided through HRSA and CDC for MCHEP (e.g., state general revenues, grants, Title X, SPRANS, CISS, etc.) is clearly necessary to support effective MCH epidemiology. As such, active pursuit of additional funds may be required of MCHEP. There is a danger of allowing grant funds to drive MCHEP activities however, and it is important to insure that grant funds compliment MCH priorities rather than supplant them.

E. Relationship with Local Health Agencies. The quality of state data, particularly program data, is often dependent on the systems used by local health agencies for data collection. Likewise, the usefulness of the data generated by the state is often measured by how well it can be used by local health agencies for program planning and policy development. Effective state MCH epidemiology involves establishing working relationships with local health agencies, assisting them to improve their own data collection and information generation, as well as increasing their capacity to utilize the information provided by the state health agency. In cases where a local health agency is more analytically advanced than the state, MCHEP should collaborate with this agency as a means of enhancing the state's own capacity, as well as part of an effort to transfer the skills and expertise of this particular local to other locals in the state. While it is possible to engage in a variety of analytic activities that do not require interaction with local agencies, if the goal is to effectively serve the MCH population throughout the state, improving data collection and use of information by the locals is essential. As such, the provision of training and technical assistance to (or with) local health agencies is a necessary component of effective MCHEP; this effort will not only increase analytic capacity within local health agencies but will build support for MCHEP throughout the state.

F. Collaboration. Collaboration outside the state health agency (e.g., universities, advocacy groups, private not-for profit organizations) should be regarded as an essential ingredient for the effectiveness of MCHEP. Collaborative activities can be used to fill resource gaps and enhance capacity for MCHEP. In addition, collaborative relationships with external stakeholders can increase investment in MCHEP; outside advocates and others can be vital partners in seeking increased resources from the state, in advocating for policy changes, and in dissemination of information about the MCH population.

G. Characteristics and Background of the Assignee. The role of the MCHEP assignee within the state is multidimensional. In addition to serving as the state MCH epidemiologist, the assignee is also called upon to perform a number of other functions including mentoring, the provision of technical assistance to other states, as well as other CDC activities. Therefore, the assignee needs a broad range of qualifications and attributes as described below:

1. Training and skills. Advanced skills (e.g., critical thinking; facility with biostatistics, epidemiology and analytic techniques; excellent oral and written communication skills) are critical to the successful assignment of an MCHEP epidemiologist. The assignee needs to have a strong background in epidemiology but should also have an understanding and if possible, some experience, in applying the skills of an epidemiologist in the public health agency setting to carry out activities such as surveillance, program planning, evaluation, and policy development. An MCH epidemiologist whose main focus and interest is etiologic epidemiology will be much less successful in integrating him/herself into the activities of the MCH planning cycle.

2. Leadership ability, management, and inter-personal skills. Critical to the assignee role is possession of both the personal disposition and management skills necessary to facilitate both mentoring and collaborative relationships. The assignee needs to function within the culture of the state, proactively forming relationships within the state health agency and with units in other agencies, and serving as a liaison between the state and CDC. A successful profile would include an individual with the skills and abilities to: (i) build collaborative relationships and partnerships, (ii) build and oversee the efforts of teams/groups, (iii) act as a team player, and, (iv) manage and/or supervise staff, as well as train and mentor staff.

3. Acculturation to CDC. CDC is a large bureaucracy with myriad protocols for its operation. Assignees must understand the expectations and responsibilities of being a CDC employee. If the assignee has not had a background in CDC’s Epidemic Intelligence Service (EIS), which has been a training ground for many MCHEP assignees in the past, or has not had a similar internship experience with CDC, the assignee needs to be appropriately acculturated to CDC and its expectations. This requires CDC staff to devote extra time to mentor these individuals to their role as CDC employees.

4. Orientation to Title V and MCH. While many assignees have training as epidemiologists and may have some background in maternal and child health (e.g., pediatrician, perinatal nurse), not all have knowledge of, or experience with state Title V programs. As such, there should be a period early on in the MCHEP assignment, in which the individual is oriented to Title V, its history, its mission, and the data and analytic requirements related to its objectives. This orientation should directly involve individuals from the Maternal and Child Health Bureau as well as the state Title V program. Such an orientation will facilitate the integration of the assignee into the work of the MCH planning cycle.

5. Match between the state and the assignee. CDC and HRSA need to consider the strengths and weaknesses of the individual assignee as well as the state before placement. For example, placement of a very junior assignee in a weak state will probably minimize the effectiveness of an assignment. While perfecting a match is often difficult, it should not be considered a trivial part of the placement process.

H. Longevity of the Assignment. While typically, CDC assignments to state health agencies are thought of as short term efforts, most MCHEP assignments have been longer. It takes a significant amount of time for an assignee to build relationships within and external to the state health agency, to understand and be able to effectively deal with the landscape of the health agency and the state political structure, to develop surveillance systems and then be able to use these systems for effective program planning and policy development, and to institutionalize the enhanced capacity that is developed. The CDC/HRSA assignment should be thought of as a long-term commitment which should terminate when enhanced analytic capacity has been institutionalized and the state can and will commit its own resources to providing enhanced epidemiologic leadership in MCH.

I. Relationship Among the Four Key Players: (MCHEP, CDC, HRSA, the state). A collaborative working relationship among the key players of the MCHEP (the assignee or designated individual responsible for MCHEP, CDC, HRSA, and the state) will promote its effectiveness. Responsibilities, lines of command, and expectations should be clearly defined. Effective lines of communication between CDC, HRSA, the assignee, the assignee’s supervisor, and the MCH/Title V director (if not the assignee’s supervisor) need to be established early on with standards and criteria for success clearly delineated. There are a number of issues to consider including:

1. Accountability. It is a challenge for the assignee to carry out all of his/her MCHEP functions while meeting the responsibility of being both a CDC employee as well as a state staff person. Because the assignee functions in dual bureaucracies, there can be confusion about ultimate decision-making authority. While accountability and effort may be clearly delineated in workplans, other issues may arise. CDC, HRSA, the state, and the assignee need to meet early on in the assignment and clearly delineate the assignee’s accountability in various relationships and how conflicts or areas of concern are to be resolved if they arise. When the major state funding partner is not Title V (e.g., Preventive Health and Health Services Block Grant dollars make up the state’s contribution), the relationship of the assignee to the Title V/MCH program must be clearly articulated up front so that it is clear this is an MCH assignment, whether or not it is Title V supported.

2. Workplan. The MCHEP required workplan should be jointly developed and monitored by the assignee in tandem with the assignee’s supervisor, the Title V/MCH director (if not the assignee’s supervisor), as well as CDC and HRSA. The workplan should be part of the state’s larger vision and mission for building MCH epidemiologic capacity building, with MCH priorities and needs driving the selected activities. As the workplan is the blueprint for the assignees’ efforts, it cannot be too general or conceptual. It should clearly define activities and projects, include a timetable (need for activity and time specificity may depend on level and expertise), and be reviewed and updated at least annually. Increased specificity will help avoid loyalty and allegiance issues that may arise.

3. CDC requirements. There are a number of job requirements established by CDC that must be clearly described at the initiation of an assignment and be understood and agreed to by the state as many of these requirements necessitate both staff time and resources. These requirements include, but are not necessarily limited to:

4. Publications. In general, CDC places a higher priority on the importance of peer-reviewed publications than do state health agencies. While most states appreciate the role of research and peer-reviewed publications, many states do not view this as a priority for enhancing MCH analytic capacity and do not necessarily fully support the assignee in this effort. On the other hand, success in this endeavor may be important for the future career of the assignee; therefore the assignee’s obligations to CDC, CDC expectations, and the state’s needs must all be considered. Of note, as the etiologic focus of these publishing efforts decreases and the emphasis on publishing results of a program evaluation or a needs assessment increases, the states may be increasingly willing to provide more support for such activities.

5. Travel. The frequency of travel required for assignees and how travel will be paid for must be clearly specified at the beginning of the assignment. The state may need or wish to designate additional funds (beyond those provided by CDC) to ensure that the assignee can attend relevant meetings and workshops.

6. Support and mentoring of the assignee. The skills and experience of the assignee must be assessed before the placement, as well as part of an ongoing process. The assignee’s needs, based both on their level of experience and previous training, should form the basis of a plan for providing ongoing support and mentoring. CDC and HRSA need to ensure that staff in the federal agencies assigned to MCHEP not only have the administrative skills to support assignees but have enhanced technical and analytic skills that also allow them to act as mentors and colleagues to the assignee.

7. Technical assistance to states as needed. It should be acknowledged that MCHEP is one component of a larger strategy to enhance MCH analytic capacity in the states. Therefore, it may be necessary for CDC and HRSA to provide additional technical assistance to states (e.g., MCH program staff, local health districts, other data units) to facilitate the work of the assignee.

8. Assignee as mentor for non-state employees. MCHEP assignees often serve as mentors for non-state employees, many of whom are CDC fellows and interns including EIS officers, and others such as preventive medicine residents or fellows. There needs to be recognition by the sponsors of these programs, by the assignee and by the state that while these programs can have short-term costs (e.g., assignee’s mentoring focus detracts from other state activities), they often also have short and long-term benefits, bringing increased capacity to the state as well as to the nation.

J. The Relationship Between CDC and HRSA/MCHB. The relationship between CDC and HRSA/MCHB (e.g., communication, expectations etc.) as the two lead federal agencies for the MCHEP program needs to be reviewed and clarified. HRSA/MCHB has a dual role in that it is both a federal and state partner in MCHEP. Although HRSA/MCHB has played an important role at the state level using state Title V funds to support assignees, in many respects, HRSA/MCHB has not fulfilled their federal leadership role in MCHEP.

HRSA/MCHB has traditionally had fewer FTE positions than CDC for placement of the assignees in states, (although now CDC is faced with the same constraint) and fewer staff assigned than CDC to provide technical support to assignees. Historically, technical assistance and support to state assignees has been undertaken almost exclusively by CDC. On the other hand, MCHB has had resources to spend on the provision of technical assistance and training to states to enhance analytic capacity (e.g., analytic training cooperative agreements to Schools of Public Health, Data Enhancement Utilization grants to states, Maternal and Child Health Information Resource Center contract, leadership training institutes, collaboration with CDC on materials to promote economic analyses); however, these efforts have generally not been coordinated with MCHEP.

HRSA’s visibility and leadership in the planning, policy and technical support aspects of this program should be increased. Specifically, HRSA should assume responsibility for the orientation of the assignee and other MCHEP staff to Title V and the MCH Block Grant. CDC and HRSA should review their respective contributions to the management of the MCHEP, and explore opportunities to strengthen their partnership to benefit the states.

The CDC/HRSA MCHEP program appears to be a very effective strategy for increasing MCH analytic capacity within state health agencies. This program deserves the strong financial and verbal support of the leadership of CDC and HRSA and should be viewed as one vital arm of a comprehensive strategy to increase the ability of state MCH programs to carry out the core functions of public health. Both agencies should seek increased funds to be able to expand such efforts to as many states as possible.

While recognizing the strength of this program, it is also important to acknowledge that there are a multitude of factors which can maximize its effectiveness. These factors must be considered as the program is expanded, since depending on the particular configuration of factors within a state, CDC and HRSA will have to invest various amounts of effort and support in the provision of technical assistance, training and mentoring.

To date, because of limited resources, MCHEP has essentially been a pilot or demonstration program. As this pilot effort has been shown to be effective, it is time to support its expansion so that all states have the necessary analytic leadership to enable them to more effectively plan and evaluate programs, as well as develop policies to improve the health status of women, children and families.