CREA Sixth International Conference


 Pre-Conference Workshops

Tuesday September 29th, 2020

Full Day 9:00am-5:00pm
Title: Transformative Mixed Methods Design
Donna Mertens, Ph.D., Professor Emeriti, Gallaudet University
Workshop description: The sophistication of transformative mixed methods designs in research will be explained and demonstrated through illustrative culturally responsive examples taken from diverse sectors and geographical regions. Transformative mixed methods designs will include applications for evaluation that supports designing a culturally responsive intervention and determining its effectiveness. Participants will have the opportunity to create transformative mixed methods designs using vignettes that reflect current social issues or a topic of their choice. 

Title: Utilization of a Culturally Responsive and Racial Equity Lens to Help Guide Strategic Engagement and Evaluation
Paul Elam, Ph.D., Michigan Public Health Institute MPHI
Mindelyn Anderson, Ph.D., Mirror Group LLC
Kristine Andrews, PhD, Child Trends
Tracy Hilliard, Ph.D., Michigan Public Health Institute MPHI
LaShaune Johnson, Ph.D., Estella Lucia Evaluation LLC
Workshop description: The field of evaluation is being challenged to utilize a process that considers who is being evaluated and who is conducting the evaluation. MPHI has worked to develop useful frameworks, tools, and approaches that evaluators could consider to focus on the ways that race and culture might influence an evaluation process; this has resulted in the development of a framework for conducting evaluation using a culturally responsive and racial equity lens. This workshop focuses on the practical use of a racial equity lens when conducting evaluation. The framework argues that culture and race are important considerations when conducting an evaluation because we believe that there are both critical and substantive nuances that are often missed, ignored, and/or misinterpreted when an evaluator is not aware of the culture of those being evaluated. Participants will be provided with a Template for Analyzing Programs through a Culturally Responsive and Racial Equity Lens, designed to focus deliberately on an evaluation process that takes race, culture, equity, and community context into consideration. Presenters will also share a “How-to Process” focused on the cultural competencies of individuals conducting evaluations, how such competencies might be improved, and strategies for doing so. This “How-to Process” is the result of thinking around developing a self-assessment instrument for evaluators, is based primarily on the cultural-proficiencies literature, and relates specifically to components of the template. Participants will have the opportunity to engage in small-group exercises to apply the concepts contained in the template to real world evaluation processes. Based on these experiences, participants will gain practical knowledge on the use of the lens.

Title: Foundations of Culturally Responsive Evaluation
Rodney K. Hopson, Ph.D., University of Illinois Urbana-Champaign
Karen E. Kirkhart, Ph.D., Syracuse University
Workshop description: This workshop addresses theoretical foundations of Culturally Responsive Evaluation (CRE) and the strategies that operationalize it in evaluation practice. It is delivered as a conversation between presenters and among participants. Following opening introductions, we will set the context with a reflection on the relevance of diversity, equity and intersectionality in the current moment of our country and where the evaluation profession sits within that. Against this backdrop, we highlight the history of CRE’s development and identify key theoretical elements. The workshop then transitions from theory to practice in three segments. The first segment pairs analysis of evaluation contexts with reflections on one’s own cultural location as an evaluator. This prepares us for the second segment, which considers methods that are culturally congruent with their contexts of practice, noting potential strengths and limitations of each. CRE values the return of benefit to the community, and the third segment examines both methods and issues in communicating findings. We pair examples from the literature with your own examples as participants to connect workshop content with your contexts, interests, and concerns. In closing the workshop, we will return to fundamental issues such as grounding CRE in social justice and how this location poses important metaevaluation questions that connect to both ethics and validity.
HALF DAY 8:00AM-12:00PM
Title: Culturally Responsive Quantitative STEM Evaluation
Olatokunbo Toks Fashola, PhD,  Research Professor American University


Workshop description: As we address this year’s theme of “Interrogating Cultural Responsiveness Against the Backdrop of Racism,” this workshop covers one of the most ethnically underrepresented academic areas:  STEM.  Over the years, state, national and private funding sources have created programs that attempt to include more minority students in STEM.  These opportunities include K-16 and beyond, after school, and informal learning settings.  The workshop also presents theoretically underpinnings that help link addresses culturally inclusive and culturally responsive pedagogy to culturally relevant evaluation.  This workshop addresses the importance of culturally relevant evaluation and assessment tools and methods specifically related to STEM.  It addresses the use of these tools in nurturing and expanding the pipeline of underrepresented minorities.  It does this by exploring constructs, theories, and methods for evaluating STEM knowledge acquisition both inside and outside the classroom in K-16 settings.  This workshop will address culturally relevant and evaluative research from a quantitative perspective. The workshop seeks to engage the workshop participants in the process of co-creating a culturally relevant topic and exploring quantitative ways to address this topic. The process will involve creating culturally relevant and quantitatively sound methods to create constructs, surveys, data dictionaries, and to administer enter, and interpret data. The outcome(s) will help to create and produce data that are not only rigorous and robust, but also data that can address topics of social justice, culturally relevant evaluation, and theories of change.
HALF DAY 1:00PM-5:00PM
Title: Re-Claiming, Re-Positioning, Re-Privileging Indigenous Knowledge Systems in Evaluation and Research


Katherine Tibbetts, Ph.D., Lili'uokalani Trust CREA-Hawai'i
Pālama Lee, Ph.D., MSW, Liliʻuokalani Trust, CREA-Hawaiʻi
Dawn Mahi, Consuelo Foundation, CREA-Hawaiʻi
Paula Morelli, Ph.D., Consuelo Foundation, CREA-Hawaiʻi


Workshop description: I ka ‘ōlelo no ke ola, i ka ‘ōlelo no ka make. Life is in speech; death is in speech. Words can heal; words can destroy. (Pukui, 1983, ʻŌlelo Noʻeau, Bishop Museum, #1191.) Equitable, transformative, culturally responsive, culturally sustaining, and Indigenous evaluation practices have many important purposes. Distilled to one value, they fundamentally promote justice and equity. This is the intention with which Kūkulu Kumuhana and the Aloha Evalution Framework were developed. This interactive workshop is designed for evaluators seeking to engage with others to explore what it means to practice evaluation with this intentionality. Kūkulu Kumuhana focuses on understanding wellbeing through a traditional Hawaiian worldview. The Aloha Evaluation Framework is centered on the practice of evaluation through the value of Aloha. In this workshop, these two frameworks are considered together as we explore how they are reshaping evaluation practice in Hawaiʻi. Through hands-on engagement with the frameworks and application of a case study, participants will: Reflect on their evaluation praxis, exploring colonial and racist assumptions when working with Indigenous groups and/or people of color, and Identify elements of these frameworks that are transplantable to their own cultural contexts and identifying what may be different in those contexts.


Wednesday September 30th, 2020

HALF DAY 8:00AM-12:00PM


Title: Acknowledging History: Enacting Community Engagement and Participatory Approaches to Foster Culturally Responsive Evaluation with Latinx Communities


Lisa Aponte-Soto, Ph.D., University of Illinois-Chicago


Workshop description: Enacting culturally responsive evaluation (CRE) with diverse Latinx communities demands evaluators employ culturally sensitive and linguistically appropriate approaches throughout all phases of evaluation. This workshop will apply CRE for working with Latinx-serving organizations to address complex issues. The session will introduce a history of social justice theories and democratic principles as a foundation to CRE. The presenter will frame culturally relevant historical contexts to highlight Latinx lived experiences, identity, and values. The facilitator will describe and demonstrate the nine-step CRE process by Frierson and colleagues (2010) using practical examples from evaluations conducted with Latinx communities. The presenter will also discuss strategies for applying community engaged and participatory approaches to guide Latinx-focused evaluations. Finally, participants will engage in self-reflection and self-awareness exercises to illustrate how our biases, prejudices, and cultural positionality influences our evaluations when working with Latinx communities.



Title: Foundations of Indigenous Evaluation


Joan LaFrance, Ed.D. (Turtle Mountain Band of Chippewa) Owner – Mekinak Consulting


Workshop description: This workshop describes the four foundational elements of an Indigenous approach to evaluation. These include recognizing the power of metaphor, grounding evaluation in Indigenous Ways of Knowing, defining core values that guide practice, and building inclusive community engagement. The foundations are based on research regarding Indigenous evaluation conducted by the American Higher Education Consortium in tribal communities and colleges. Each element draws from Indigenous practices and ways of knowing such as storytelling, community engagement, and relationship. The foundations serve as guideposts for evaluators who seek cultural/tribal relevant approaches to evaluation. During the workshop, each of the foundations will be described and discussed. The workshop is designed to engage participants in the foundational elements through small group exercises that encourage application of the elements to local tribal and community situations.



Title: LGBTQ+ Evaluation and Cultural Responsiveness: An Intensive Workshop


Dylan Felt, B.A., Northwestern University
Gregory Phillips II, Ph.D., Northwestern University
Erik Elías Glenn, MSW, Northwestern University
Josh Boegner, MPH, B.A., Northwestern University


Workshop description: In her 2017 CREA Conference keynote, Dr. Robin Miller called for the field to do better in accounting for LGBTQ+ populations. Thanks in great part to Dr. Miller’s work, recent years have seen growing recognition of the importance of culturally competent evaluation practices with LGBTQ+ populations. However, many evaluators still lack the tools and training to do this work well. In recognition of this, this session will serve as an intensive training for evaluators new to or moderately experienced in LGBTQ+ evaluation, and will provide attendees with the skills they need to conduct thoughtful, critical, intersectionally-minded work with LGBTQ+ populations. Through an interactive, intensive workshop, attendees will learn, discuss, and practice applying the essential principles of LGBTQ+ evaluation. Our agenda prioritizes a collaborative dynamic, as attendees will learn together and from each other, and will be encouraged to integrate perspectives from their own experiences and practices to ground all learning in the real world. Through blending theory, practice, and praxis in this intensive workshop, attendees will emerge with the tools they need to be culturally responsive LGBTQ+ Evaluation Champions in their own practices, and LGBTQ+ allies in their everyday lives.



Title: Perspectives on Fairness and Equity in Educational Assessment


Drew Gitomer, Ph.D., Rutgers University 
Emi Iwatani, Ph.D., Digital Promise


Workshop description: Approaches to addressing fairness and equity in assessments have been considered by those both inside and outside of the measurement community.  Each of these approaches brings with it very different lenses and assumptions about the meaning and operationalization of fairness and equity in assessments as well as the very function of assessments in supporting larger goals of education. Therefore, the meanings of the terms fairness and equity are not shared or understood among measurement and educational communities.  This workshop will clarify their different meanings and associated implications for practice and policy. This workshop will include four segments that consider how fairness and equity are conceptualized and acted upon from the perspectives of psychometrics, educational preparation, culturally responsive assessment, and critical theory.   Each section will first introduce these perspectives.  Small groups will then consider a particular assessment context from that perspective.  Participants will gain an appreciation of different perspectives and better understand how they intersect and diverge.