Developing Construct-Valid Psychological Scales:

From Theory to Measurement

This three-day workshop provides a practical, theory-driven introduction to developing construct-valid psychological scales. Participants will learn how to move from construct definition and literature review through item pool development, structural analyses (exploratory and confirmatory factor analysis), and item response theory to build psychometrically sound measures. Through live demonstrations and applied exercises, students will develop and evaluate their own item pools and learn how to assemble evidence for convergent and discriminant construct validity.

Register
Register

Instructor:
Leonard J. Simms, PhD (University of Buffalo)

Workshop Format:
Three-Day Live Online Workshop

Workshop Dates and Times:
July 8–10, 2026
(10:00am to 3:30pm ET)

Video Availability:
New Workshop for 2026!
New Videos Available on July 11, 2026

This workshop provides a comprehensive, theory-driven introduction to developing psychological scales that demonstrate strong construct validity, grounded in the framework articulated by Jane Loevinger, Lee Anna Clark, and David Watson. The workshop begins with a review of core psychometric principles and the role of measurement in theory development, emphasizing how careful construct identification, literature review, and attention to content validity form the foundation of effective scale development. Participants will learn how to define constructs clearly, map their theoretical boundaries, and translate those definitions into well-designed item pools that adequately sample the intended construct domain.

The second portion of the workshop focuses on evaluating the internal structure of an item pool using modern psychometric tools. Participants will learn how exploratory and confirmatory factor analytic techniques can be used to identify the latent structure of items, refine item pools, and develop interpretable and reliable scales. Instruction also will introduce the role of modern measurement models, including item response theory, for evaluating item functioning and improving scale precision. Throughout the workshop, live demonstrations will illustrate each analytic step using real data, allowing participants to see how psychometric decisions are made during the iterative process of scale refinement.

Finally, the workshop will address how to assemble evidence for construct validity by situating new measures within a broader nomological network. Participants will learn how to evaluate convergent and discriminant validity, test theoretical predictions, and interpret patterns of relations with external variables to support the intended meaning of a scale. The course is designed to be interactive and applied, with hands-on exercises and homework assignments that guide participants through each stage of scale development. By the end of the workshop, participants will have developed a preliminary item pool for a construct of interest and a clear plan for evaluating and validating a new psychological measure.

What you’ll learn

  • Define and operationalize psychological constructs by conducting focused literature reviews, mapping construct boundaries, and translating theory into well-specified measurement domains.

  • Develop high-quality item pools by applying principles of content validity, facet mapping, and effective item writing to ensure adequate coverage of the construct domain.

  • Evaluate and refine scales using modern psychometric methods, including exploratory and confirmatory factor analysis and item response theory to assess dimensionality, item performance, and scale structure.

  • Establish construct validity by situating new measures within a nomological network and evaluating convergent and discriminant validity.

Syllabus

Day 1 – Foundations of Construct-Valid Scale Development

  1. Overview of Psychometric Foundations: reliability, validity, dimensionality, and measurement error.

  2. Construct Identification and Definition: clarifying the construct, theoretical boundaries, and relations to adjacent constructs.

  3. Literature Review and Existing Measures: evaluating prior theory and measurement approaches.

  4. Content Validity and Domain Sampling: building content validity into item pools to ensure strong relevance and representativeness.

  5. Item Pool Development: writing clear, representative items and organizing them into construct facets.

Day 2 – Structural Evaluation of the Item Pool

  1. Preparing Item Pools for Analysis: sampling considerations, pilot testing, and preliminary item diagnostics.

  2. Inter-Item Relations and Dimensionality: examining correlations and identifying redundant or poorly functioning items.

  3. Exploratory Factor Analysis in Scale Development: factor extraction, rotation, and interpreting latent structure.

  4. Confirmatory Factor Analysis: testing hypothesized structures and evaluating model fit.

  5. Iterative Scale Refinement: selecting items and refining scales based on structural evidence.

Day 3 – Advanced Psychometric Models and Construct Validation

  1. Introduction to Item Response Theory: evaluating item discrimination, difficulty, and information.

  2. Comparing Classical and Modern Measurement Models: when to use CTT vs. IRT in scale development.

  3. Building a Final Scale: balancing reliability, coverage, and interpretability.

  4. Evaluating Convergent and Discriminant Validity: situating the scale within a nomological network.

  5. Planning Validation Studies: designing studies to accumulate evidence for construct validity and scale utility.

Registration Options

Developing Construct-Valid Psychological Scales

  • Professional
  • Baseline Price for Faculty,
    Staff, and Other Professionals
  • Click Register Below
  • Trainee
  • 33% Discount for
    Students and Postdocs
  • Use code "TRAINEE" at Checkout

 FAQs

  • This workshop is designed for graduate students, postdoctoral scholars, and researchers in psychology and related behavioral sciences who are interested in developing or evaluating psychological measures. It will be particularly useful for individuals conducting research that requires the creation of new scales or the refinement of existing instruments to assess psychological constructs. Although the workshop includes a brief review of basic psychometric concepts, participants should have a general familiarity with psychological research methods and an interest in applying quantitative methods to measurement development.

  • Beginner to Intermediate

  • Although the workshop includes a brief review of basic psychometric concepts, participants should have a general familiarity with psychological research methods and an interest in applying quantitative methods to measurement development.

  • We will use a combination of SAS, SPSS, Mplus, and R

  • Slides, examples of code, and mock data sets will be provided to facilitate homework and post-hoc learning.