Designing and Conducting Generalizable Psychological Research

This workshop is designed to provide a guide for strategies to developing, planning, and collecting for social, behavioral, and medical sciences for the purpose of generalizing findings across populations and for specifically understanding disparities. The course will focus on discussing best practices for assessment measures, strategies for increasing sample representation and generalizability, and in-depth discussion of considerations when collecting data from underrepresented samples.

Instructor:
Craig Rodriguez-Seijas, PhD (University of Michigan)

Workshop Dates and Times:
Thursday, August 7, 2025 at 9:30am-3:30pm EDT
Friday, August 8, 2025 at 9:30am-3:30pm EDT
(And 2024 Recording is Available Now)

Workshop Format:
Two-Day Online Workshop and Recording

Psychological, and other related scientific disciplines, are built upon data that is overwhelmingly representative of certain populations, with serious considerations of generalizability often included as an afterthought. With many scientists’ increasing interest in being able to collect representative populations and additional focus being paid to diversifying research samples (e.g., increasing scrutiny of samples from journals and reviewers), being able to appropriately represent the wealth of experiences within our research samples is imperative. However, increasing the generalizability of research samples might not always be straightforward to all researchers and can be influenced by multiple factors. Similarly, approaching the design of research projects with generalizability in mind can also lead to interesting data analysis decisions and comparisons.

This 2-day course is designed to provide the basics of appropriate theory and strategies researchers can employ when designing studies to maximize representation and generalizability of their research samples. Topics covered will include: Theory to inform research planning, Considerations for measure selection and questionnaire design, Psychometric considerations, Recruitment strategies for representative samples, Common challenges, and troubleshooting. Time will also be set aside to troubleshoot specific research projects participants are planning, thus allowing participants to consider strategies that will be appropriate for them prior to data collection efforts begin.

What you’ll learn

  • Theoretical Considerations – Understand how sociocultural context might impact variables of interest and how to build this into the research design and protocol. Appreciate how science as usual excludes specific samples to the field’s detriment.

  • Survey Design – Learn the types of questions to ask and what you can ask. Expand your understanding of considerations that can inform question and measure selection and design.

  • Recruitment & Participant Engagement – Learn strategies that can benefit representativeness of study samples from leveraging pre-existing resources for data collection to tips from experiences with designing and collecting data from different samples.

  • Personalized Feedback – Access personalized feedback from the instructor for participants’ specific research projects and to troubleshoot individual concerns about how to increase diversity of research samples.

Syllabus

Day 1 – Conceptual Foundations

  1. Why generalizability?

  2. Philosophy of Psychological Sciences

  3. Contextual Theories that Undergird Data Collection & Analysis

  4. Measurement Considerations: Demographic Identities

  5. Moving Beyond Demography

Day 2 – Practical Application

  1. Best Practices for Collecting Demographic Information

  2. Recruitment Strategies for Enhancing Sample Generalizability & Accessing Difficult-to-Reach Populations

  3. Retention of Participants

  4. Statistical Procedures of Interest

    • Latent Variable Assumptions & Measurement Selection

    • Factorial Invariance Approaches

    • Differential Item Functioning & Item Response Theory

  5. Individualized Feedback & Project Planning

Registration Options

Designing and Conducting Psychological Research with Diverse Samples

  • Professional
  • $499
  • Baseline Price for Faculty,
    Staff, and Other Professionals
  • Click Register Below
  • Trainee
  • $499 $334
  • 33% Discount for
    Students and Postdocs
  • Use code "TRAINEE" at Checkout

Note: All registration options for this workshop come with three things:
(1) Access to the video recording and materials of the 2024 version of the workshop until August 7, 2025
(2) The ability to attend the live recording of the 2025 version of the workshop on August 7-8, 2025
(3) Access to the video recording and materials of the 2025 version of the workshop after August 8, 2025

If this workshop is offered again in future years (e.g., 2026+), then you will have continued “evergreen” access to the new recordings and materials.

 FAQs

  • This workshop is specially designed for researchers in the behavioral and health sciences who are interested in learning strategies to help them design studies to maximize sample generalizability.

  • Beginner

  • No specific prior knowledge is necessary. Basic research design experience is recommended.

  • Participants will be provided with a curated reading list and article pdfs.