American Evaluation Association 2018 Conference CREA Professional Development Workshops

October 29 - November 3
Cleveland, OH
 
Workshops Summaries October 30,Tuesday

Foundations of Culturally Responsive Evaluation (#65)
Tuesday Full Day           

Presenters: Rodney Hopson, PhD., Professor University of Illinois Urbana-Champaign
                   Karen Kirkhart, PhD., Professor, Syracuse University

Workshop Description: This workshop addresses theoretical foundations of Culturally Responsive Evaluation (CRE) and the strategies that operationalize it in evaluation practice. It is delivered as a conversation between presenters and among participants. Following opening introductions, we will set the context with a reflection on the relevance of diversity, equity and intersectionality in the current moment of our country and where the evaluation profession sits within that. Against this backdrop, we highlight the history of CRE’s development and identify key theoretical elements.

The workshop then transitions from theory to practice in three segments. The first segment pairs analysis of evaluation contexts with reflections on one’s own cultural location as an evaluator. This prepares us for the second segment, which considers methods that are culturally congruent with their contexts of practice, noting potential strengths and limitations of each. CRE values the return of benefit to the community, and the third segment examines both methods and issues in communicating findings. We pair examples from the literature with your own examples as participants to connect workshop content with your contexts, interests, and concerns. In closing the workshop, we will return to fundamental issues such as grounding CRE in social justice and how this location poses important metaevaluation questions that connect to both ethics and validity.

Presenters Biographies:

 

Rodney Hopson

RODNEY K. HOPSON is Professor, Department of Educational Psychology, College of Education, University of Illinois – Urbana Champaign. 

He received his  Ph.D. from the Curry School of Education, University of Virginia and has done post-doctoral/sabbatical studies in the Faculty of Education, University of Namibia, the Johns Hopkins Bloomberg School of Public Health and Centre of African Studies, Cambridge University. He served as President of the American Evaluation   Association in 2012.
Dr. Hopson’s research interests lie in social politics and policies, foundations of education, sociolinguistics, ethnography, and evaluation. His work raises questions that 1) analyze and address the differential impact of education and schooling on marginalized and underrepresented groups in diverse global nation states and 2) seek solutions to social and educational conditions in the form of alternative paradigms, epistemologies, and methods for the way the oppressed and marginalized succeed and thrive despite circumstances and opportunities that suggest otherwise.

 

karen Kirkhart

KAREN E. KIRKHART is currently Professor and Director of the Baccalaureate Program, School of Social Work, David B. Falk College of Sport and Human Dynamics, Syracuse University. Dr. Kirkhart is also an affiliated faculty member of the Center for Culturally Responsive Evaluation and Assessment (CREA) at the University of Illinois, Urbana-Champaign. 

She has been active in the evaluation profession for over thirty-five years. She holds a Ph.D. in Social Work and Psychology from The University of Michigan,Dr. Kirkhart served as President of the American Evaluation Association in 1994. Her leadership has been recognized with the Robert Ingle Award for Outstanding Service to the AEA. Dr. Kirkhart’s scholarly contributions to multicultural validity, cultural competence in evaluation, and evaluation influence have been recognized by the AEA with the Paul F. Lazarsfeld Award for Outstanding Contribution to Evaluation Theory.


Utilization of a Racial Equity Lens to help Guide Strategic Engagement and Evaluation (#66)
Tuesday Full Day           

Presenters: Paul Elam, Ph.D., Chief Strategy Officer, MPHI
                   LaShaune Johnson, Ph.D., Assistant Professor, Creighton University
                   Mindelyn Anderson, Ph.D., Program Director, American University
                   Willard Walker, Public Policy Associates, Inc.

 

Workshop Description: The field of evaluation is being challenged to move from the traditional role of evaluation, and its perceived role of objectivity, to a process that considers who is being evaluated and who is conducting the evaluation.  Over the past three years, Public Policy Associates, Inc. (PPA) has worked to develop useful frameworks, tools, and approaches that evaluators could consider to focus on the ways that race and culture might influence an evaluation process; this has resulted in the development of a framework for conducting evaluation using a racial equity lens.

This workshop focuses on the practical use of a racial equity lens when conducting evaluation.  The framework argues that culture and race are important considerations when conducting an evaluation because we believe that there are both critical and substantive nuances that are often missed, ignored, and/or misinterpreted when an evaluator is not aware of the culture of those being evaluated.

Participants will be provided with a Template for Analyzing Programs through a Culturally Responsive and Racial Equity Lens, designed to focus deliberately on an evaluation process that takes race, culture, equity, and community context into consideration.  

Presenters will also share a “How-to Process” focused on the cultural competencies of individuals conducting evaluations, how such competencies might be improved, and strategies for doing so.  This “How-to Process” is the result of thinking around developing a self-assessment instrument for evaluators, is based primarily on the cultural-proficiencies literature, and relates specifically to components of the template.

Participants will have the opportunity to engage in small-group exercises to apply the concepts contained in the template to real world evaluation processes.  Based on these experiences, participants will gain practical knowledge on the use of the lens.

• Share strategies and tools to intentionally include a culturally responsive and racial equity lens in strategic program evaluation.


Presenters Biographies:

 

Paul Elam

Paul Elam, Ph.D.,Chief Strategy Officer, MPHI

He is a skilled researcher with expertise in justice issues.  He is a collaborative leader who brings an abiding commitment to diversity, inclusion, and equity to his public policy work.  Dr. Elam has a wealth of knowledge and experience measuring racial and ethnic discrimination and believes that sound public policy analysis should include an examination of whether all people are being treated fairly and equitably.  Dr. Elam directed a state-wide evaluation of the W.K. Kellogg Foundation Michigan Team’s investments and used the Template for Analyzing Programs through a Culturally Responsive and Racial Equity Lens as part of the evaluation design. 
Dr. Elam recently began assisting the Annie E. Casey Foundation with the Expanding the Bench Initiative.  This initiative aims to improve evaluation science and social innovation by increasing diversity in the field of research and evaluation.  Dr. Elam’s work will focus on evaluators from historically underrepresented groups with evaluation expertise in the areas of child welfare and juvenile justice.  Dr. Elam is also a certified facilitator for the California Brief Multicultural Competence Scale (CMBCS) Multicultural Training Program.

 

Willard Walker

Willard Walker, Affiliated Consultant 

He has extensive experience working with workforce development, state policy, school-to-work, and race and diversity issues.  Mr. Walker has provided training on issues of inclusion, equity, and diversity for the Lansing Board of Water & Light; coordinated an Employer Conference on Workplace Diversity in the Lansing area; and assisted in a crime analysis evaluation for the Michigan Department of Human Services, Bureau of Juvenile Justice.  Mr. Walker led the work of developing the Template for Analyzing Programs through a Culturally Responsive and Racial Equity Lens in his work for the W.K. Kellogg Foundation Michigan Team Evaluation.
He has worked with clients to improve recruitment and retention of African-American teachers, including development of a coordinated recruitment planning process for urban districts.  He conducted research to address the plight of young males of color in a project initiated through the Joint Center for Political and Economic Studies.

 

LaShaune Johnson,

LaShaune Johnson, Ph.D., Assistant Professor, Creighton UniversityWillard

She is an experienced researcher of online learning among pre-health professional students, breast cancer disparities, adult and adolescent obesity, and pediatric health literacy among immigrant/refugee populations.  She is faculty in the Master of Public Health program, and in the Master of Medical Anthropology program.  She is currently the co-chair of the Metro African American Breast Cancer Task Force in Omaha, Nebraska, and is the co-director of the “In Search of a Medical Home”, a culturally sensitive, Muslim community-based educator project in Central Missouri, funded by the American Academy of Pediatrics and the Society for Community Research and Action.  She is one of the contributors of a recently published textbook, Social Capital and Community Well-Being, which is being used by the Serve Here Connecticut academic debt reduction program.  She is also one of the contributors to an upcoming edited volume on Black LGBT health issues.         
Dr. Johnson has employed novel community-based methods to support Omaha’s Adolescent Health Project developmental evaluation (participatory video) and another method (Photovoice) to investigate health services for obese patients in Connecticut and Nebraska.  For the Breast Cancer Task Force, she co-designed a peer educator/advocate program; this program is in its second year and is expanding to add training for patient navigators.  She is currently a member of the Building Healthy Futures Evaluation Advisory Board in Omaha.  She was a member of the inaugural Annie E. Casey Foundation LEEAD (Leaders in Equitable Evaluation and Diversity) program.   

 

Mindelyn Anderson

Dr. Mindelyn Anderson currently serves as the Program Director of the Masters of Science in Measurement and Evaluation at American University and is the founder of Mirror Group LLC.

Most recently, she completed a Leaders in Equitable Evaluation and Diversity (LEEAD) Fellowship at the Annie E. Casey Foundation. Her previous higher education appointments include Honors Faculty in Residence and Assistant Professor of Sociology and African American Studies at Northeastern University and Marilyn Yarbrough Fellow at Kenyon College. Dr. Anderson has also held fellowships at American Institutes for Research and Mathematica Policy Research.
For 15 years, Dr. Anderson has researched social inequality and stratification, race and migration, education and social mobility. She has conducted evaluations with community-based, regional, national and international non-profit, for-profit, and educational organizations. Her utilization-focused, participatory evaluation practice is informed by Culturally-Responsive Evaluation and values diversity, equity, and inclusion as integral components of high quality evaluation. Her scholarship has been supported by sponsors including the National Science Foundation and National Endowment for the Humanities.

Draft Workshop Agenda

 

9:00–9:20 A.M. - WELCOME AND INTRODUCTIONS

  • Introductions and Session Learning Objectives
  • Workshop Agreements
  • Context for the Workshop

 

 9:20–10:00 A.M. - PART I: INTRODUCTION TO THE RACIAL EQUITY LENS TEMPLATE

  • Overview of Culturally Responsive and Racial Equity Lens
  • Poll Group
  • Small Group Discussion

 

 10:00–10:50 A.M. - PART II: SELF-ASSESSMENT FOR EVALUATORS

  • Instructions
  • Assignment: Complete Assessment
  • Discuss in Triads
  • Small Group Report Out and Discussion
  • Compare Results with Trends in the Field

10:50–11:00 A.M  -  BREAK

11:00–11:50 A.M. -  PART III: EVALUATION EXERCISE TO APPLY WHAT YOU HAVE LEARNED

  • Instructions
  • Assignment: Design an Evaluation Using a Culturally Responsive & Racial Equity Lens
  • Small Group Report Out and Discussion

11:50–12:00 P.M.      WRAP-UP

  • Final Comments
  • Resources
  • Workshop Survey

 

Workshops Summaries October 31,Wednesday
Indigenous Evaluation: Foundations, Building Relations, and Applications  (#67)
Wednesday Half-Day AM           

Presenters:    

Nicole Bowman (Mohican/Lunaape), Ph.D., Evaluator & Researcher,  University  of Wisconsin-Madison
Carolee Dodge-Francis (Oneida), Ed.D., Associate Professor/Executive Director American Indian Research & Education Center- University of Nevada-Las Vegas
Andrea Guerrero-Guajardo (Cherokee/Tejano), Ph.D., MPH, University of the Incarnate Word School of Osteopathic Medicine

Workshop Description: Indigenous evaluation (IE) is a member of the culturally responsive evaluation family with important intersectional applications to western evaluations.  This practical and interactive workshop will facilitate learning and understanding of theories and methodologies associated with Indigenous evaluation to be utilized for applied practice. Workshop participants will be given strategies to build authentic relationships, professional practice matrixes, and examples of how to co-design effective system, program, or project evaluation studies to collaborate with. Specifically, workshop participants will be instructed on how to conduct with (not on) Tribal nations and Indigenous communities. The premise of this session is based upon the four directions contained within a medicine wheel.  Participants will experience whole brain engagement (i.e. linear, mini-lecture as well as creative, experiential, cultural, and group activities); gain knowledge with regard to activities and pedagogical strategies: cultural and community protocols gathered through traditional knowledge sources; theoretical, legal, and methodological foundations of Indigenous evaluation; alignment and relationship to western theories, methods, and design; and application to real world (and your world) demonstrations, case studies, and community of practice discussions.  Participants will leave with many new resources and supportive relationships to ensure the direction of Indigenous evaluation, and you as an evaluator!


Presenters Biographies:

 

Nicole Bowman

Waapalaneexkweew Nicole Bowman (Mohican/Lunaape-Munsee), PhD, President, Bowman Performance Consulting, Researcher/Evaluator, LEAD & WEC Centers, University of WI-Madiso

Dr. Nicole Bowman is the president and founder of the nationally award-winning Bowman Performance Consulting (BPC) in Shawano, Wisconsin and a culturally responsive Evaluator at the University of WI-Madison’s Center for Education Research. She is from the Mohican and Lunaape (Delaware) Tribal community and is Lynx Clan.  Her spirit name is Waapalaneexkweew (wah-pah-lah-nay-wook; Flying Eagle Woman) Neeka ha (knee-kah-ha; she is) Newetkaski (Nah-wet-tah-kah-see; accompanied by) Newa (nay-wah; four)  Opalanwuuk (oh-pah-lun-knee-aye-wook; Eagles).  Waapalaneexkweew Neeka Ha Newetkaski Newa Opalanwuuk is her full name. She’s been a proud resident of northeastern Wisconsin (Packer/Cheesehead Country) for over four decades and happy spouse to David Farrell for almost 25 years.
Dr. Bowman earned her PhD in Educational Leadership & Policy Analysis (ELPA) at the University of Wisconsin-Madison (UW-Madison) in 2015, where she was an Academic Fellow and the first Indigenous woman to get her PhD from UW’s ELPA department and only the 11th American Indian to get a PhD from UW-Madison’s School of Education in over 150 years.  Her dissertation is recognized as the nation’s first multi-jurisdictional educational systems policy study in the country to systemically examine how Tribal and non-Tribal educational policy is developed and implemented. Through her work and networks at BPC and UW-Madison, she provides culturally responsive evaluation, research, and policy subject matter expertise where Tribal and non-Tribal governments and organizations collaborate.  As the president of BPC, Dr. Bowman has contributed over two decades of culturally responsive and multi-jurisdictional evaluation, research, training, and technical assistance.  BPC’s living mission and motto is "working WITH people and not ON them." This ethos has contributed to Dr. Bowman’s reputation as a responsive and respected subject matter expert on multi-jurisdictional, education policy, and Indigenous evaluation.  Dr. Bowman currently represents AEA on the International Work Group and on EvalIndigenous in the EvalPartners' global initiative to connect evaluation efforts around the world and is an evaluation and policy journal reviewer for numerous national and international publications. This year she is the 2018 AEA Robert Ingle Service Awardee.  She is also a proud Research Affiliate of the Center for Culturally Responsive Evaluation and Assessment (CREA) at the University of Illinois – Urbana Champaign. 

 

 

Carolee Dodge-Francis

Carolee Dodge-Francis (Oneida), Ed.D., is Executive Director of the American Indian Research and Education Center, and Associate Professor within the School of Community Health Sciences

She is an American Indian researcher, and nationally recognized for her research, program development and evaluation work in chronic disease within Tribal communities. She has over 20 years experience in community/public health, health education development and research with urban and rural American Indian communities nation-wide. Her funding awards include the National Institutes of Health, Center for Disease Control and Prevention, Nevada State and private foundations.

 

 

 

Andrea Guerrero-Guajardo

Andrea Guerrero-Guajardo (Cherokee/Tejano), Ph.D., MPH, Director Community Engagement and Social Accountability at the University of the Incarnate Word School of Osteopathic Medicine. 

Her approach to community-based medical education is firmed entrenched in the principles of community organizing and participatory decision making with the goal of developing local capacity and improved wellbeing. Her community-based work is focused on social determinants of health and community responsive evaluation that addresses disparities caused by social, political, and economic inequities within historically oppressed and marginalized populations. Dr. Guerrero-Guajardo works collaboratively with stakeholders at the local, state, and national levels to develop prudent public health policy and promote population health. She is an experienced trainer in these topics and was recognized by the Obama White House for her expertise and contributions to best practices for coalition building and community collaboration. Her leadership roles within the American Evaluation Association (AEA) supported Multi-Ethnic Issues in Evaluation and Latino Responsive Evaluation Discourse, and she was selected to be a 2013 Minority Serving Institutions (MSI) fellow at AEA. Dr. Guerrero-Guajardo lives in San Antonio with her husband, Ben, and their four children.

 

 

 

Culturally Relevant Evaluation and Research from a Quantitative Perspective (#68)
Wednesday AM           

Presenter: Toks Fashola, PhD., Professor, American University

Workshop Description: This workshop will address culturally relevant and evaluative research from a quantitative perspective. The workshop seeks to engage the workshop participants in the process of co-creating a culturally relevant topic and exploring quantitative ways to address this topic.
The process will involve creating culturally relevant and quantitatively sound methods to create constructs, surveys, data dictionaries, and to administer enter, and interpret data.
The outcome(s) will help to create and produce data that are not only rigorous and robust, but also data that can address topics of social justice, culturally relevant evaluation, and theories of change.
The workshop will use some examples of projects that currently exist, and projects that are in progress. Workshop participants will be encouraged and guided to become informed consumers of culturally relevant quantitative research.


Presenter's Biography:

 

Olatokunbo (Toks) S. Fashola

Olatokunbo (Toks) S. Fashola (Ph.D.) is a Research in Residence / Research Professor at American University, and a consulting Principal Research Scientist and Vice President for Evaluation at Mathematics Education Research Associates (MERA)

She has expertise in randomized field trials (RFTs) and Quasi Experimental Designs and has served as principal investigator, evaluator, and advisor for several programs and program evaluations across the country. Her current research interests include STEM among underrepresented students in grades K-16, Resiliency theory, Resiliency theory in specialized settings, bilingual education and language and literacy acquisition, opportunities to learn during the non-school hours, culturally relevant program evaluation, and educating African American Males.  She is also interested in college attendance and dropout prevention programs that work, early childhood education, and wraparound programs.  Dr. Fashola uses mixed methods to conduct her research, which includes policy analysis, program evaluation, and culturally relevant evaluation and assessment.

 

 

EmPOWERING Latinx Communities: Practicing Culturally Responsiveness Evaluation with Latinx Lens  (#69)
Wednesday PM           

Presenter: Lisa Aponte-Soto, University of Illinois-Chicago

Workshop Description: This workshop will focus on translating contemporary culturally responsiveness for actionable evaluation practices for and with Latinx communities. Latinos are the fastest growing population in the United States, accounting for 16.3% of the total population (2010 census). Enacting culturally responsive evaluation (CRE) with diverse multinational, racial, and ethnic Latinx communities demands highly skilled evaluators who can employ evaluation approaches which align and support diverse perspectives in all evaluation phases. The session will begin with a brief history of social justice oriented evaluation theories, CRE, and Latino Critical Race Theory (LatCrit). This paradigmatic framing will provide a foundation to discuss the nine-step CRE process in action with Latinx communities. Facilitators will highlight synthesized literature and draw on their own indigenous praxis-oriented perspectives. Participants should come prepared to ‘dig deep’ and share their experiences with Latinx-focused evaluation planning and practice


Presenter's Biography:

 

Lisa Aponte-Soto

Lisa Aponte-Soto (Ph.D.) serves as Associate Director of Community Based Research for the Office of the Vice Chancellor for Health Affairs at the University of Illinois at Chicago, Mile Square Health Center (MSHC)

Mile Square Health Center (MSHC) is network of 13 Federally Qualified Health Centers providing comprehensive, high quality health services through the continuum of care. In this capacity, Dr. Aponte-Soto coordinates a portfolio of community-based research projects, oversees the MSHC patient screening and navigation programs, and conducts evidenced-based program, process, and outcomes evaluation to monitor progress, mitigate issues on an ongoing timely basis, and ensure continuous improvement.
Formerly, Dr. Aponte-Soto served as National Program Deputy Director of the Robert Wood Johnson Foundation New Connections program at Equal Measure. Lisa is committed to the professional development and advancement of racial and ethnic students and scholars from underrepresented backgrounds through education, advising, and career coaching. She describes herself as an advocate, researcher, and evaluator promoting culturally responsive practices to foster academic and health equity among underserved, underrepresented and vulnerable populations. Lisa has 20 years of experience leading public, private, and federally funded diversity initiatives. In these capacities, she has conducted mix-methods and community-based research and evaluation applying CRE principles. Lisa is an alumna of the American Evaluation Association (AEA) Graduate Diversity Internship Program. She is also a founding member and chair of the AEA Latina/o Responsive Evaluation Discourse Topical Interest Group (TIG), a member of the AEA Task Force on Engagement, Diversity, and Leadership Development, a member of the AEA Guiding Principles Task Force, a former program co-chair of the AEA Multiethnic Issues in Evaluation TIG, and a former member of the AEA Cultural Competence Working Group.