The California Wellness Foundation
HEALTH IMPROVEMENT INITIATIVE:
Challenges, Accomplishments
and Lessons Learned

By Alicia Procello and Gary Nelson

TABLE OF CONTENTS
1 Introduction to the Health Improvement Initiative
2 Public Education and Policy Program
3 Health Partnership Program
4 Initiative Support Program
5 Conclusion

Accomplishments, Challenges and Lessons Learned

In January 1995, The California Wellness Foundation’s (TCWF) board of directors approved a five-year $20 million grantmaking program titled The Health Improvement Initiative (HII). The overall goal of the HII was to improve the health of Californians by increasing public recognition of the importance of factors, other than health care, in determining the health of California residents and by building new collaborative approaches to improve population health. Population health recognizes that the health of all of us in turn affects the health of each of us. Such elements as the neighborhood’s history, levels of employment, types of housing available to residents, services, policies and the safety of neighborhoods are all relevant in this broader population health context. Specific goals of the HII included: promote awareness of the broad determinants of health; shift the current focus and investment in health toward prevention; establish comprehensive, integrated systems of preventive services; and demonstrate that population health improvements can occur in real-world settings.

The HII consisted of three integrated components: the Public Education and Policy Program, the Health Partnership Program and the Initiative Support Program. Grantmaking in the Public Education and Policy Program focused on public opinion polling, nonpartisan policy analysis, policy education and public education about population health. The Health Partnership Program, the centerpiece of the HII, enabled 15 geographically diverse California community/county-based health partnerships to design and implement local health improvement plans. The health partnerships were diverse in terms of history as collaboratives, governance structure, lead fiscal agency, health problems addressed and populations served. The Initiative Support Program was responsible for building a learning community in support of population health improvement. This was accomplished through the systematic organization and delivery of technical support services. Finally, the Initiative evaluator was charged with enhancing the efforts of the health partnerships and supporting grantees through a continuous quality improvement evaluation and assessing HII implementation and impact.

This report outlines the accomplishments, challenges and lessons learned during the HII’s five-year period that ended in June 2001. Goals and objectives are presented for each portion of the Initiative followed by staff analysis of what was actually accomplished. This report is organized according to the goals and objectives of the Health Improvement Initiative. At the conclusion of the three components, we reflect upon the accomplishments, challenges and lessons learned about the Initiative as a whole.

Alicia Procello and Gary Nelson, Program Directors
The California Wellness Foundation

Goal:

Provide public education and policy development in support of community-based efforts to promote population health.

Objectives:

• Support public education regarding population health issues.
• Support nonpartisan policy analysis regarding key public policy issues in the state.
• Provide information about population health to policymakers and opinion leaders.

Accomplishments:

• The establishment of a state-level policy center. Through the establishment of a state-level policy center, the HII made significant contributions to improved public/policymaker awareness—a prerequisite to population-based health improvements. Activities such as statewide public opinion polls and corresponding publications, creation of policy development tools, ongoing media advocacy and nonpartisan technical assistance for elected officials established a statewide leadership role in advancing a broad view of health and its determinants.
• Contributions to state-level policies that emphasize prevention. Evidence of the Initiative’s achievement can be found in the HII grantees' role in California’s Healthy Families Program, Proposition 10, parity in mental health insurance coverage, tobacco prevention and control, alcohol and drug treatment, and school-based health and physical education. Contributions to each of these state-initiated programs have included the presentation of data (scientific and public opinion research) that have been translated into legislative language or as guidelines for implementing state-level policy. Critical to this success, HII grantees did an outstanding job of “diagnosing” emerging policy issues.
• Unanticipated state-level program and policy contributions. As a result of the HII grantees, the Initiative has made numerous unplanned and/or unintended contributions to prevention. The HII has provided grant resources relevant to population health and health improvement; participated, represented or voiced support for health improvement in other venues; supported the development, acquisition or utilization of tools; produced and/or disseminated reports; and provided technical assistance and support for population health improvements.
• The use of state and local public opinion polling for planning and policy development. The HII provided new information about the public’s perspective on prevention issues. HII grantees generated state and local public opinion data that were instrumental in policy development, education and advocacy. Six state-level and three local health improvement polls provided a new source of information on public perceptions of need, demand and potential solutions to current health issues. Collectively, these efforts have generated national interest among government, nonprofit and philanthropic organizations.
• Policy publications: a new source of information on prevention. The production and broad dissemination of policy briefs represented a significant contribution of the HII to public and stakeholder awareness of prevention issues. Over time, HII grantees mastered the art of designing brief action-oriented policy tools that presented compelling policy messages. Approximately 30 policy briefs/tools were published that ranged from financing prevention, systems reform, results-based accountability, civic engagement, alcohol and drug treatment, welfare reform, early child brain development, health and physical education, to smart growth and urban sprawl. The use of electronic communications, professional organizations and the print media enhanced the success of this overall effort.

Challenges: 

• Language describing the process and intended outcomes of the HII was not always clear, consistent or widely understood by multiple audiences. Concepts such as population health or systems change lack a shared meaning. This was a challenge among HII grantees as well as those served by the Initiative. In many respects, these concepts were “ahead of the curve” of public opinion and understanding.
• Few audiences outside of public health understand, embrace or practice population health. The shift from a personal health care orientation to a population perspective among service providers and other sectors of the community has not occurred on a broad scale. More effort was needed to engage state and local elected officials. The HII was unable to invest additional time and resources in education and relationship building to address this challenge.
• Policy development, education and advocacy efforts were compromised by insufficient capacity and/or competing priorities. Many health partnerships were challenged to actively engage their communities in policy debate and policy change. Some health partnerships appeared to lack an understanding of the local opportunities (versus state level) to create a favorable policy environment. In contrast, HII grantees that addressed statewide advocacy issues such as nonpartisan analysis, use of data, strategic alliances, media advocacy and publication strategies were far more successful than the local health partnerships grantees. The skills set required to address policy may be quite different from the skills required to implement other aspects of the health partnership goals/objectives. As a consequence, technical support made available was not initially utilized, and “early” policy successes for the health partnerships may have been delayed or lost.
• The impact of specific policy education and advocacy efforts was difficult to establish. While statewide Initiative grantees made important contributions, it was difficult to quantify the policy achievements and establish attribution particular to local health partnerships. Short of legal policies enacted through legislation, the HII was challenged to identify compelling intermediate indicators of accomplishment. Methodological limitations in the evaluation of policy change also affected our understanding of statewide contributions.
• A nonpartisan statewide prevention platform embraced by elected officials is noticeably absent. It was evident that an expanded view of health was not widely understood or championed at the policy level. In addition, there appeared to be little political will to attack the cynicism among those who question the value of prevention. The political environment, term limits of elected officials and the expertise of elected officials and their staffs represented obstacles to a nonpartisan, state-level commitment to prevention.
• A state-level policy center faces challenges as a neutral convenor providing nonpartisan analysis, while refraining from advancing specific proposals. Although HII grantees were effective in each of the aforementioned roles, frequently the advocacy role needed to be passed on to other entities to maintain trust among stakeholders and comply with lobbying restrictions.

Lessons Learned:

• Statewide policy development, education and advocacy were important strategies and critical to the achievements of the HII. The HII structure and theoretical framework for population health improvement is validated by the accomplishments of a state-level policy center. A Public Education and Policy Program enhances the success of comprehensive strategies, such as the HII, which are designed to change structures, processes and norms. However, the prevention message still needs to be delivered loudly, boldly and frequently.
• Establishment of a state-level policy center was perhaps the single most important contribution of the HII to statewide population health improvement efforts. TCWF’s decision to establish a statewide presence in prevention policy will be one of the important legacies of this Initiative. A strong nonprofit organization has been created, its core functions are well developed and its role within the state has been widely embraced. Sustainability of its efforts appears promising. Overall, the policy center’s presence has leveraged resources, influenced policymakers and provided leadership in the prevention arena.
• Statewide health improvement public opinion surveys were innovative and effective HII strategies. Public opinion data provided a new source of information about need, demand and direction for future health improvements in California. The translation of public opinion data into policy briefs timed in response to policy decisions provided an important but previously missing piece of information. While most HII public opinion surveys were timely and relevant to current policy issues, some surveys (e.g., civic engagement) and related publications appeared to be ahead of current thinking and practice. Finally, sampling and the development of representative samples among selected racial/ethnic populations proved difficult.
• The development of policy briefs was a highly successful strategy to informing policy development efforts. Through the work of the HII grantees, the development of strategic policy publications based on nonpartisan analysis proved to be a critical resource to decisionmakers and elected officials. The development of electronic methods of communicating information about model policies also proved beneficial. This publication strategy was exemplary and should be more widely adopted among policy advocates.
• The absence of a strategic and comprehensive public and stakeholder education campaign limited the potential policy contributions of the Initiative. In retrospect, the accomplishments of the HII Public Education and Policy Program were not fully realized because of the lack of well-choreographed public education campaigns. As designed, the HII did not include provisions for public education campaigns.
• Policy development, education and advocacy were differentially achieved among the HII health partnerships. Intervening factors may have been related to the scope and scale of health partnership efforts; the health problems addressed; mission and experience of the collaborative; constraints of the fiscal agent; and staff expertise. For some health partnerships, policy efforts resulted in identifiable successes; for others achievements are forthcoming; and for others they will be unrealized. Among health partnership grantees, policy development, education and advocacy appeared to be an “add-on” or an afterthought. In retrospect, efforts in organizing health partnerships for the work of the Initiative, (i.e., establishing the collaboratives, providing direct services and undertaking systems change efforts) delayed their policy work. TCWF staff attempted to correct this imbalance by the mid-point of the Initiative; however, for some collaboratives the initial work seemed too formidable to invest more resources into policy. It may be that engagement in policy work is developmental and a characteristic of more experienced, mature community collaboratives.
• Technical support was needed to advance local policy development. Among health partnership grantees there was considerable variability in skill and experience in policy development. Most health partnerships did not fully utilize the technical support available, while others benefited significantly. Data collection, analysis and reporting for purposes of policy development represented a common technical assistance need.

Goal:

Provide resources and technical assistance to selected communities for the purpose of structural and functional enhancements in programs and policies designed to promote population health.

Objectives:

• Support model community-based public/private partnerships that ensure access for all populations, including the most vulnerable, to core public health services.
• Support a restructuring of organizations and/or resource allocations to address improved population health.
• Provide direct services including new programs and policies that contribute directly to improvement in population health.

Accomplishments:

• The establishment of multisector collaboratives to address comprehensive, integrated preventive services. The Health Partnership Program was composed of 15 collaboratives representing California’s geographic diversity. Cohort 1 represented nine health partnerships that addressed five primary objectives: direct preventive services, systems change, population health measurement, governance and sustainability. Cohort 2 represented four collaboratives that addressed direct preventive services and one of the five primary objectives above. Cohort 3 represented two health partnerships that addressed capacity building.
• The provision of preventive health services. Residents in participating HII communities now have improved access to direct preventive services. Among Cohort 1 grantees, the number of preventive health services increased substantially over the life of the Initiative, with 17,855 high-intensity services provided (e.g., case management, immunization and mentoring); 40,488 medium-intensity services (e.g., screening, counseling, HIV testing, consulting and referrals); and 36,040 low-intensity services (e.g., dental care and cleaning, in-home safety checks and health education).
• System and service integration of preventive services. By the end of the Initiative, Cohort 1 health partnerships fully implemented 21 systems change initiatives in five sites and nine systems change initiatives were still in process. Systems change is defined as permanent changes in the way the systems of preventive services are organized and delivered in the community that have potential to lead to population health improvements. Within the HII, systems change activities were grouped into four major areas: service integration, results-based budgeting, data integration and policy development.
• Population health measurement and policy development. Each Cohort I health partnership completed three local health improvement surveys and four sites subsequently published and/or advanced policy directives. Seven Cohort 1 and two Cohort 2 health partnerships also developed community/county-level report cards intended to track long-term health outcomes. The process and outcomes of each health partnership’s workplan also benefited from formative evaluation feedback provided by the Initiative evaluation.
• The development and recognition of local leadership. Each health partnership functioned as a collaborative with stable response patterns in governance satisfaction, decisionmaking and the establishment of new community linkages as reported by the overall HII evaluation governance surveys. Resident involvement and leadership recognition was achieved through the public acknowledgement of Public Health Champions within each health partnership. The program recognized and acknowledged the role of community residents in identifying the systems issues in need of change and their involvement to assist in the change process to affect population health.
• Leveraged resources and sustainability. Continuation of the health partnerships’ work was a major objective of the HII. By the end of the Initiative, a range of sustainability options were being implemented including the transition from collaboratives to nonprofit organizations, involvement in new philanthropic initiatives, and the procurement of new financial resources to support aspects of the health partnership workplan that continue population health improvement efforts. Collectively, the work of the health partnerships generated an additional $6.67 million in grants and contracts.

Challenges:

• Achieving systems change is hard work. Systems change or reform takes time, occurs at many levels (e.g., neighborhood, agency, city, county) and represents a fundamental change in orientation toward service delivery. While the language may be confusing, it was equally difficult to determine where reform should begin—integration of services, integration of data, finance reform, or new organizational structures and relationships? A sequential approach to systems change with a clearly defined pathway leading to efficient and effective delivery of preventive services was needed to assist health partnerships. Systems change efforts undertaken by health partnerships lacked access to promising models or best practices that could have been instructive.
• Authentic community involvement is difficult. It was difficult to engage residents “at the table” within predominantly agency-based health partnerships. Each site acknowledged the critical importance of community engagement to achieve systems change, but experienced challenges accommodating grassroots demands and expectations in systems change and policy objectives. Being inclusive of community participation is essential but can also unduly dictate the agenda. Issues such as cultural competency, primary languages spoken, abstract concepts and jargon and meeting locations/time all affected authentic community involvement. There was also tension within health partnerships regarding the methods of resident involvement versus resident-driven approaches to population health improvement. As the HII progressed, the definition of community involvement expanded to reflect the uniqueness of each health partnership.
• Attribution for advances in population health is difficult to determine in community settings. It is difficult to attribute success and failures within population-based collaboratives where simultaneous multiple efforts affect city or countywide health outcomes. TCWF staff, health partnerships and HII evaluators desired greater confidence in perceptions of attribution and accomplishment. In the final analysis, the question became: “What contributions did the health partnership make to observed results?”
• Each health partnership’s lead agency/fiscal agent affected the process and outcomes of grant activities. Accountability to the Foundation, to the lead agency’s board of directors and to the health partnership often placed the lead agency and collaborative members in competition and/or conflict. Issues such as organizational turf and power were played out in most health partnerships. In most cases, the lead agency/fiscal agent seldom assumed an equal partnership role in collaborative efforts -- leading to resentment of the lead agency, disillusionment of health partnership members, competition and/or apathy in seeking additional resources.
• The role and contributions of the local evaluators were often not realized. Each health partnership allocated funds to support a local evaluator and all but one site developed and implemented local evaluation plans. Some local evaluators adapted well to the HII focus while others did not. Some were responsive to health partnership needs; others pursued their own agendas or relied on a skill set not compatible with the health partnership. With few exceptions, the local evaluator was not an asset in the design, implementation and publication of local public opinion surveys.
• Outcomes didn’t always matter. There was general frustration that the health partnerships’ performance and improved outcomes did not always matter to stakeholders. As the work of the HII progressed, it appeared that positive results did not routinely affect policy, inform decisionmaking, impact funding allocations, etc. Conversely, some decisions were made in absence of, or prior to, results being reported. This issue deserves further attention, as each participant in the HII is susceptible to ineffective and inefficient efforts if there is not a commitment to learning and using results -- good or bad. Initiative evaluations must begin with a commitment to using results, and health partnerships must be committed to use these results to inform current efforts and future investments.

Lessons Learned:

• Underserved populations continue to be disadvantaged by ineffective, inefficient, categorical and fragmented programs and services. The social inequalities that affect underserved populations are further compounded by dysfunctional approaches to organizing and delivering services. Multifaceted strategies that incorporate direct services, policy, public education, capacity building and evaluation define successful population health improvement efforts.
• Community collaboration and partnerships are challenging, labor intensive and evolutionary, yet their role in population health improvement is essential. To sustain a population health approach, health partnerships needed to create and sustain the dialogue within their communities. This assumes that planning is a continuous process and does not end when implementation begins. Health partnerships must continue to establish new alliances and engage in continuous planning to advance current and future population health approaches. Preliminary evidence of success demands that health partnerships continue providing leadership and stay the course to improve population health.
• Strong individual and organizational leadership is characteristic of productive, high-performing programs. Throughout the Initiative, indigenous leaders stepped forward to become champions for the public’s health. Leadership recognition can be a powerful strategy to mobilize a collaborative. Extraordinary individuals warrant recognition and continued support.
• Support local health partnership staff as much as reasonably possible. Health partnership staff members are typically caught in a struggle to balance substantial program and public responsibilities against largely insufficient levels of authority within a collaborative. This disparity raised the level of health partnership staff turnover, affecting institutional memory and collaborative commitment. Health partnership staff turnover made it difficult to tailor technical support. As staff would move in and out of key positions within the collaborative, modifications in technical support (content and delivery) were required. Continued HII orientations were necessary throughout the five-year period. Coaching and professional development provided through technical support helped avoid greater staff turnover.
• Maximize flexibility in program planning and implementation. Over five years, there were many unforeseen obstacles and opportunities. Collaboratives must have flexibility to adjust accordingly and remain relevant in a dynamic real-life context. Timing, good luck, perseverance and the ability to seize opportunities as they emerge play as large a role as anticipation and planning.
• Phased funding of health partnerships was difficult to integrate into a comprehensive learning community. The establishment of Cohorts 2 and 3 supported health partnership applicants not initially recommended due to funding limitations. These grants extended the geographical penetration of the Initiative to include collaboratives that were not focused on systems change activities. The accomplishments of Cohorts 2 and 3 were significant in the delivery of direct preventive services and targeted systems change activities. However, differences in scope and scale of funding and workplans between cohorts minimized the participation of Cohorts 2 and 3 in technical support, training and other HII convenings.
• Absence of an outcomes orientation can dampen enthusiasm. Community-based health collaboratives have not been performance/outcomes-oriented, and it was difficult to sustain energy and involvement with abstract or long-term goals. Too often, the absence of larger victories gave the impression of futility and there was little patience with incremental change and modest improvements. It was essential to celebrate even the smallest accomplishments, or “low hanging fruit,” to maintain a sense of purpose, camaraderie and commitment.
• Evaluation can be a burden. Some health partnerships perceived evaluation activities as invasive. Technical support was frequently provided in response to the burden of evaluation activities. It was difficult to strike a balance between the evaluation needs of the Initiative and the needs of each health partnership. The benefits of participation in the Initiative-wide versus local evaluation were not apparent to some. In addition, the HII community needed to make better use of the evaluation data obtained by documenting, reporting and disseminating results. Publication and dissemination of results is time consuming and resource intensive. Planning should begin early and consideration given to a variety of venues to reach target populations.
• Population health measurement is an important strategy to help communities focus on results. Prior to the start of the HII, there was a recognition that new measures of the health of a community (e.g., report cards or surveys) were needed to raise awareness of health outcomes, the costs of bad results, the assets of a community and the undervalued role of prevention. Data were seen as a means to educate the public and policymakers and shift the investment towards prevention. Those health partnerships that demonstrated the greatest progress in this area have stronger foundations for policy and systems change as well as greater community support.
• Linking integrated preventive services to systems change was challenging. A goal of the HII was to link direct preventive services to systems change activities. While health partnerships made progress in the development and provision of preventive health services, it was challenging to link those services to systems change efforts if it was not well designed originally. Those sites that were successful in linking systems change with service integration appeared to be more successful in implementing their workplans. Overall, the impressive efforts by health partnerships to deliver preventive services were underrecognized in the HII compared to systems change activities.
• Dedication to achieving population health improvement is more than just a grant and the money. For many health partnerships, population health improvement efforts existed prior to the HII and the Foundation’s resources enabled them to pursue or operationalize existing efforts. Some health partnerships considered the grant and the resources a diversion from a more fluid community assessment process that did not lend itself to systems change as defined by the Initiative. Other health partnerships were simply drawn to the resources associated with the Initiative and varied in their success in achieving population health improvements.
• The scope and scale of the health partnerships’ efforts affected outcomes. Some health partnerships took on very difficult and comprehensive issues whereas others worked much more narrowly. Consequently, the problems and the solutions for some were much more complicated, and it was more difficult to determine success within five years. Compounding these issues were: the variety of local neighborhood issues; breadth of the target neighborhoods, communities or counties; rural or urban settings and associated resources; and difficulty in measuring population health indicators in five years. Overall, it was appropriate to allow each health partnership to determine its own scope and scale of workplan activities because it honored a local process.

Goal:

Provide technical assistance and research support for programs and policies that promote population health.

Objectives:

• Develop and implement methods of measuring population health of communities – e.g., composite health indices and report cards.
• Develop standards for evaluating programs and policies that promote population health.
• Build the capacity of communities to make informed decisions and resource allocations in support of population health through increased opportunities for training and technical assistance.

Accomplishments:

• A sustainable learning community. A shared perception by Foundation staff and HII grantees is that the Initiative was responsible for creating an environment conducive to professional growth, problem solving, renewal and sustainability. Throughout the Initiative, a climate existed where grantees freely shared information, pursued common goals and/or collaborated to solve problems. What will be sustained are the relationships, the experience of working in an environment where communities assisted each other and the experience of effectively providing peer support.
• Implementation of a responsive, multifaceted, technical support delivery system. The HII logic model included provisions for technical support/training that were flexible, responsive and specific to each health partnership. The lead technical support grantee adopted a brokering model of technical support that effectively utilized the skills of health partnership grantees and many outside organizations. Technical support included a variety of support services and delivery mechanisms including: grantee technical assistance, training workshops, electronic communications, peer support convenings, technical assistance tools and publications. Throughout the five-year Initiative, HII grantees were involved in providing continuous feedback about the effectiveness of technical support services received as well as future support needed. The Foundation’s utilization of Initiative-wide management meetings and discussions with HII support grantees contributed to a responsive technical support system.
• The effective integration of evaluation into the HII. One of the important accomplishments of the HII was the effective integration of evaluation into the delivery of technical support and subsequent enhancements in the performance of HII grantees. All too often, evaluation and technical assistance compete for the attention of grantees. The evaluation strategy was simultaneously responsive to each HII grantee and the overall HII. Noteworthy contributions of the HII evaluation included: documentation of health partnership governance and implementation, definition and assessment of systems change, creation of indicators of success and establishment of broad interest in the evaluation of community-based health initiatives.

Challenges:

• The provision of technical support by multiple grantees can be limiting. The lead technical assistance provider adopted a brokering model of technical support that worked very well but limited to some extent the technical support contributions of other Initiative grantees.
• It is difficult to balance Initiative-wide management activities with overall technical support activities. Throughout the Initiative, convenings for technical support occurred two to four times a year. While attempting to meet the technical support needs of grantees, management issues regarding the design and implementation of the HII were raised and required time to address. In retrospect, more management meetings designed to facilitate greater coordination and joint decisionmaking regarding the overall Initiative would have been preferred.
• Efforts to produce technical support tools fell short of initial expectations. At the outset, TCWF grants were awarded to identify best practices in community collaboration and to develop a Population Health Index to assist health partnerships in measuring the health of their communities. These tools were intended to assist in the planning and implementation of systems change workplans. In both instances, self-promotion took precedence over the grantees’ attempts to provide timely and effective technical support to the Initiative. In some circumstances, contracts and other procurement mechanisms may be preferred so as to allow the Foundation to retain ownership and control of products/services.
• It is difficult to evaluate the impact of technical support/training. How should technical support be evaluated? Should it be focused on process or outcomes? Is consumer satisfaction sufficient? If capacity, productivity or performance are the desired outcomes, how are these evaluated in such a way that technical support can be identified as a contributing factor? The HII struggled with an inadequate science base from which to assess the impact of technical support. It was not well understood how the provision of technical support services benefited members of the health partnerships and the communities. Technical support grantees did an outstanding job of exposing various members of the partnerships to training and technical support; however, the HII had no formal strategy by which these individuals would share skills/lessons with others in the community.
• Over time, greater technical support was needed to address organizational and partnership development issues rather than issues particular to systems change. As the health partnerships entered into the second year of the Initiative they began to experience a variety of managerial/administrative challenges including employment policies, budgeting, staff supervision and interorganizational relationships. Increasingly, the technical assistance provider focused on management-related technical support rather than systems change technical support.
• Inadequate time and resources were provided for technical support activities. Further work is needed to establish a permanent system of support and services to health partnerships and other community initiatives that are responsive to the scope and scale of the assistance required. Throughout the HII, the need for technical support was often greater than the health partnerships’ demand, despite repeated offers of assistance by support grantees and encouragement to use technical support by TCWF. This was particularly true of support for systems change, policy development and population health measurement.

Lessons Learned:

• A learning community (providing mutual support, technical assistance and training across sites) enables collaboratives to solve problems, capitalize on opportunities and sustain their efforts. Future improvements in population health demand further investment in capacity-building efforts directed at individuals, organizations, collaboratives and communities.
• The evaluation of community-based health initiatives requires a focus on four broad innovations and promising directions: identifying a theory of change, focusing on building local evaluation capacity for continuous improvement, promoting trust and communication, and making outcomes matter. Community-based approaches have a growing appeal in the search for effective ways to improve the health of populations. The appeal is tempered, however, by challenges in implementing and evaluating this approach. Individual, organizational and community capacity are all realistic and important outcomes of community-based initiatives. Evaluations of these initiatives have yet to fully establish capacity as an asset and an important evaluation outcome.
• A theory of action improves planning, implementation and evaluation. The Initiative Support Program validated the HII theory of action and was well served by the model. The logic model or theory of action also helped TCWF staff in conceptualizing the grantmaking program and focus of the Initiative evaluation. There appeared to be a direct relationship between the use of logic models by health partnerships and clarity in goals, objectives and workplans.
• There are multiple ways of defining and enhancing sustainability. In addition to financial resources, the HII established that systems, structures, processes and capacities also potentially contribute to the health partnerships’ sustainability efforts. Health partnerships benefited from their participation in the HII as their work embraced other initiatives and contributed to the sustainability of other programs. The HII appeared to be the common denominator or infrastructure for many subsequently funded local initiatives. Not surprisingly, health partnerships participating in national, state or local initiatives competed very well for resources—an indicator of sustainability. Yet, it is clear that discussions of sustainability should begin early. Health partnerships need to assess the value of the collaborative, direct services and systems changes prior to seeking continued funding. Therefore, research on sources of funding and compatibility with collaborative efforts should be considered during program planning.
• Initiative management can be exercised in multiple forms. The HII was well served by an external advisory committee convened for joint purposes of providing TCWF staff with guidance and with serving as a two-way information link to other state and national efforts. Ongoing management meetings with HII grantee representatives promoted overall coordination and a learning community. Management support provided to health partnership project coordinators promoted problem solving and a continuous feedback loop to the technical support provider and to TCWF. Large-scale convenings generated local grantee support and offered the advantage of broader dissemination. The HII mid-Initiative conference was a catalyst for many grantees that required validation, if not recognition for their efforts. The designation of a grantee to serve as convener and facilitator of management support was critical to this effort.
• A technical support brokering model was effective. Technical support needs to be flexible, site specific, proactive and customized. Isolating technical support from evaluation and funding responsibilities affords the technical support provider space for confidentiality and neutrality. This model ultimately proved more effective and efficient than alternative technical assistance delivery systems.
• Participants should be engaged in the design and implementation of technical support services. The participant-driven agenda and peer-to-peer learning emphasis of technical support convenings contributed greatly to their success and utility. The multiple avenues for providing input on technical support activities (e.g., surveys, participation in design committees, evaluations) helped to foster an environment of continuous quality improvement.
• Knowing when to intervene. Over time, Foundation staff acquired an understanding of when and under what conditions the funder’s decisions take precedence. While Foundation staff designed and implemented the Initiative with a strong value placed on partnerships, this was occasionally in conflict with other values including results, accountability and respecting community voice. Giving health partnerships decisionmaking authority made participation possible that might otherwise be denied in a more autocratic “strong handed” administration of the HII. That is not to say that conditions, sanctions and direct, honest feedback were not considered when individual grantee efforts strayed from the broadly defined mission, goals and objectives of the HII.
• Funders must “walk the talk.” Finally, TCWF staff concludes with the sine qua non of responsible grantmaking. As we have learned, funders of comprehensive community-based health initiatives must be prepared to walk the talk when advocating for partnerships, collaboration, reform or accountability among grantees. Leading by example enhances the prospects of grantee success.

The California Wellness Foundation’s Health Improvement Initiative was a commitment to improve population health, an outcomes based way of understanding the health and quality of life of large numbers of people. At the conclusion of this investment the question must be asked, was population health within the state of California improved? It was clear at the onset that the Initiative’s five-year time frame would yield few measurable population health improvements. Instead, the evaluation captured intermediate outcomes, concentrating on systems change at the community level and the provision of direct preventive health services. Overall, grantees in the Public Education and Policy Program and the Health Partnership Program found the most success addressing systems change on two levels: service integration – providing comprehensive, integrated services responsive to the needs of the community, and policy development – developing and implementing new policies that promote population health. Health promotion and/or disease prevention direct services were provided to 94,383 individuals or groups. Most of these services will be sustained beyond the HII funding period. Grantees of the Initiative Support Program made important contributions to the Initiative by creating a learning community where relationships, peer support and dedication to health improvement will be sustained.

The Health Improvement Initiative was an important undertaking to change the emphasis from a personal health care orientation to a population perspective among service providers and other sectors of the community. As we look to the future, the HII’s lessons learned have been extended to both state and national forums, and we believe the population health debate, both in California and nationwide, will continue to be influenced by HII publications, grantees and those who contribute to population health improvement efforts. The Foundation thanks the dozens of organizations and hundreds of individuals who made the Health Improvement Initiative an exciting, productive, engaging and informative experience. It is our hope that all those involved with the Initiative will continue to work toward improving population health in California.