Evaluation Plan
A comprehensive evaluation plan will support development and implementation of Designing Our Tomorrow deliverables. Throughout the project, evaluation activities will help monitor the degree to which the team is engaging in continuous improvement, accountability, and knowledge-building, and will inform potential improvements to project deliverables. Research components of the project will be evaluated through a Research Oversight Committee. Finally, a summative evaluation of deliverables will be conducted by an independent, external evaluator: Rockman et al. The logic model above represents a summary reference version of the more detailed Designing Our Tomorrow logic model (not included due to size) to be further refined during the early phases of the project. Overall evaluation will be coordinated at OMSI and conducted in collaboration with Rockman et al. Evaluation will be conducted in English and Spanish based on participant preference and will draw from best practices in culturally responsive research and evaluation (AEA 2011; Allen et al. 2007; Frechtling 2002; Okazaki and Sue 1995).
Front-end, Formative and Remedial Evaluation
Aligned with the project logic model, front-end, formative, and remedial evaluation studies will be conducted to support and assess deliverables for OMSI visitors, Adelante Mujeres families, Fleet Science Center visitors, and professional audience members including exhibit developers, museum educators, and researchers. OMSI staff will lead front-end, formative, and remedial evaluation studies with support from Rockman et al, including assistance developing and revising instruments and protocols and feedback on data collection and analysis.
External Oversight of Research Activities
A Research Oversight Committee consisting of three experts in the ISE field will monitor and assess the degree to which the research studies meet the criteria of being rigorous, reliable, trustworthy, and culturally responsive, and will provide ongoing feedback to inform the research team’s decision-making. The committee will meet regularly with the project team via conference and video calls, communicate with the project team via email and phone as needed, and contribute to a yearly report to NSF. Feedback throughout the project will help inform team decision making, while the written report will provide documentation of the extent to which the project is appropriately making progress towards its intended goals.
Summative Evaluation
Rockman et al will lead summative evaluation for all public audience deliverables. Data collection will include observations of, brief exit interviews with, and tracking of participation and exhibit usage by family groups both at OMSI and the Fleet Science Center. Data collection from the professional audiences of exhibit developers and designers and educators will include surveys and interviews and ongoing review of professional audience documents and deliverables