split ballot experiments

• Solutions Industry Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Member Experience Technology Use case AskWhy Communities Audience InsightsHub InstantAnswers Digsite LivePolls Journey Mapping GDPR Positive People Science 360 Feedback Surveys Research Edition
• Resources Blog eBooks Survey Templates Case Studies Training Webinars Help center

Split Ballot Technique

Divide the respondents into various groups and collect the same information with slightly different questions. reduce the effect of position-bias with the split ballot technique..

Join over 10 million users

What is split ballot technique?

The split ballot technique is a survey method where a group of respondents are divided in half (or several smaller groups). Each group receives a questionnaire that asks for the same information but using slightly different questions.

Using a half split technique can help reduce the effects of position bias in multiple-choice survey questions. For example, a survey may ask a question that lists answers as:

In another version of the survey, respondents may be asked the same questions with answers listed as:

The question hasn’t changed, but survey takers may have a subconscious bias based on the position of the answer. Some respondents might be prone to choosing the first answer more often than other responses, even if they’re not aware of it.

Other ways split surveys reduce potential for bias is to use experimental manipulation. This takes two versions of the same question to assess participants’ willingness to express opinions. For example, one set of surveys may use the phrase “greater than” listed first, while the other set lists “less than” first. Having two differently worded questions that ask for the same answer helps reduce potential biases.

Although basic questions are asked of all participants, the split half technique in research may have slightly mismatched data when comparing certain variables to total completed surveys. Since some survey respondents had different questions than the other half, you could have a difference between the number of survey participants and the totals for some of the variables.

For example, if you have two surveys of 5 questions each, question number 5 may have two versions for the two surveys:

When calculating the total value of respondents who choose “1. X” in your survey, you’ll have fewer responses than the number of surveys you sent out. Half of your participants did not see the same question with that answer available.

QuestionPro uses a split ballot technique to measure the effects of two frames: regulation versus non-regulation. A secondary manipulation, varying the order of schemata measures (frame first versus schema measures first), tests the role of schemata, or behaviors, in framing effects. Split ballot technique allows you to try out different versions of the same questions to see which version seems to result in the most accurate reporting. It also ensures that the response frequencies will be stable enough for you to draw the necessary conclusions. It allows researchers to make causal interpretations and helps them to be more certain about the reasons for any effects they obtain.

One example of using the split half technique in research is to find the general public’s opinion on a topic. Three versions of the most important question on an issue are used in agenda-setting research to measure issue salience, or level of importance, among the public. A split ballot in a state-wide survey compared multiple versions of the public agenda with a social frame of reference versus a personal frame of reference, the traditional term problem versus issue, and the effects of question order. High correlations between the different versions were found in all three sets of comparisons.

What is Survey Router / Split Ballot Testing?

Split Ballot Testing is when you randomly divide your sample into two or more sub-samples and perform an experiment. The experiment could be different versions of a survey, different surveys altogether, or a combination of these. The goal is to see if any differences exist between the versions of the survey or how users behave differently.

How do I set up Survey Router / Split Ballot Testing?

Select the Router Mode and assign surveys and click on the Save Changes button.

A URL will be provided which can be used for distribution to your respondents or sample panel. Depending on the mode selected the surveys will be displayed.

What are the different router modes?

Target Survey: With target survey you can set up a URL and assign a survey for the URL.

Split Ballot: With split ballot you can set up a URL and assign Multiple Surveys . Randomly one of the selected surveys will be picked.

License Restrictions

This feature/tool [Survey Router / Split Ballot Testing] is not available as part of any of our standard self-service licenses. It is part of our Team Edition License. Please contact your Account Manager for pricing and options for purchasing the Team Edition License.

Please contact sales to learn more about Team Edition

• Sample questions
• Sample reports
• Survey logic
• Integrations
• Professional services
• Survey Software
• Customer Experience
• Communities
• Polls Explore the QuestionPro Poll Software - The World's leading Online Poll Maker & Creator. Create online polls, distribute them using email and multiple other options and start analyzing poll results.
• Research Edition
• InsightsHub
• Survey Templates
• Case Studies
• AI in Market Research
• Quiz Templates
• Qualtrics Alternative Explore the list of features that QuestionPro has compared to Qualtrics and learn how you can get more, for less.
• SurveyMonkey Alternative
• VisionCritical Alternative
• Medallia Alternative
• Likert Scale Complete Likert Scale Questions, Examples and Surveys for 5, 7 and 9 point scales. Learn everything about Likert Scale with corresponding example for each question and survey demonstrations.
• Conjoint Analysis
• Net Promoter Score (NPS) Learn everything about Net Promoter Score (NPS) and the Net Promoter Question. Get a clear view on the universal Net Promoter Score Formula, how to undertake Net Promoter Score Calculation followed by a simple Net Promoter Score Example.
• Offline Surveys
• Customer Satisfaction Surveys
• Employee Survey Software Employee survey software & tool to create, send and analyze employee surveys. Get real-time analysis for employee satisfaction, engagement, work culture and map your employee experience from onboarding to exit!
• Market Research Survey Software Real-time, automated and advanced market research survey software & tool to create surveys, collect data and analyze results for actionable market insights.
• GDPR & EU Compliance
• Employee Experience
• Customer Journey
• Executive Team
• In the news
• Testimonials
• Advisory Board

QuestionPro in your language

• Encuestas Online
• Pesquisa Online
• Umfrage Software
• برامج للمسح

Awards & certificates

The experience journal.

Find innovative ideas about Experience Management from the experts

• © 2021 QuestionPro Survey Software | +1 (800) 531 0228
• Privacy Statement
• Terms of Use
• Cookie Settings

• Split-ballot Design in Surveys: Meaning, Applications, Pros & Cons

Introduction

Split-ballot design is a valuable technique employed in survey research to reduce bias and increase the validity of survey results. It involves dividing the survey sample into multiple groups and presenting each group with different versions of the survey questionnaire. 

This approach allows you to explore the impact of variations in question formats, stimuli, or other experimental conditions on participant responses. In this article, we will delve into the concept of split-ballot design, its applications, and its pros and cons. We will also provide a comprehensive understanding of this vital survey research technique.

Understanding Split-Ballot Design

Split-ballot design refers to the practice of creating multiple versions of a survey questionnaire and randomly assigning participants to one of the different versions. The purpose of employing a split-ballot design is to examine how variations in the survey instrument affect participant responses. Thereby allowing you and other researchers to explore potential biases and increase the validity of your findings.

For example, you may present one group of participants with a question using a Likert scale , while another group receives the same question with a semantic differential scale . When you compare the responses between these groups, you can gain insights into how the choice of question format influences participant perceptions and preferences.

Split-ballot designs offer flexibility in the types of variations that can be introduced. These variations can include alterations in question-wording, response options, question order, or the inclusion of different stimuli. For instance, you can present participants with different versions of an advertisement. This can include an image, or video to explore how these stimuli affect attitudes, perceptions, or behavioral intentions.

Employing split-ballot designs can help you uncover potential sources of bias in your survey research. Additionally, split-ballot designs enable you to assess the robustness and generalization of your results, as you can explore the consistency or variation of findings across different experimental conditions.

Advantages of Split-Ballot Design

Split-ballot designs offer several advantages and benefits in survey research. Here are some key advantages:

• Control for response bias and order effects : Split-ballot designs allow researchers to control for response bias and order effects. By randomly assigning participants to different versions of the survey questionnaire, the impact of response biases, such as acquiescence bias or primacy/recency effects, can be assessed. This helps to minimize the influence of these biases on the overall results, leading to more accurate and reliable findings.
• Compare and contrast effects of different variables or conditions : The designs enable researchers to compare and contrast the effects of different variables or experimental conditions. For example, you can examine the impact of different pricing options, product features, or advertising messages by presenting different versions of the survey to different groups. This allows for a direct comparison of participant responses under varying conditions, providing valuable insights into the factors that drive preferences or behaviors.
• Enhance validity and generalizability: When you explore the impact of different survey designs or experimental conditions, split-ballot designs help you increase the validity and generalizability of survey findings. Researchers can identify design choices or variations that may introduce bias or affect responses, allowing for adjustments and improvements to enhance the overall validity and reliability of the survey instrument. Additionally, assessing the consistency or variation of findings across different experimental conditions provides a more comprehensive understanding of the phenomenon under investigation.

Implementing Split-Ballot Design

Implementing a split-ballot design requires careful planning and consideration. Here are some key steps involved:

• Define research objectives and variations: Clearly define the research objectives and identify the specific variations or conditions to be tested in the split-ballot design. This could include variations in question-wording, response options, stimuli, or experimental conditions.
• Randomly assign participants: Randomly assign participants to different versions of the survey questionnaire to ensure unbiased representation. Randomization will help you to minimize selection bias and ensures that each participant has an equal chance of being assigned to any particular version.
• Consider sample size and statistical power: Adequate sample sizes are essential to ensure sufficient statistical power and validity of the findings. Consider the desired effect size, significance level, and statistical power calculations when determining the sample size for each group. Larger sample sizes provide greater precision and increase the reliability of the results.
• Implement consistent data collection procedures : Use standardized data collection procedures across all versions of the survey to maintain consistency. Ensure that the administration of the survey, instructions provided to participants, and data collection protocols are the same across all groups. This will help you to minimize potential confounding factors.
• Analyze and compare results: Analyze the data collected from each version of the survey questionnaire and compare the results across the different groups. Use appropriate statistical techniques, such as hypothesis testing or regression analysis, to assess the effects of the variations on participant responses.
• Interpret and report findings: Interpret the findings of the split-ballot design, considering the impact of the variations on the survey results. Discuss the implications of the findings, including any significant differences observed between the groups. You must report the split-ballot design and its implementation details transparently to allow for replication and to enhance the credibility of the study.

Examples and Applications of Split-Ballot Design

Split-ballot designs have been widely used in various research domains to investigate the impact of variations on survey responses. Here are some examples of split-ballot designs in different fields:

• Political polling: In political polling, split-ballot designs are often employed to test the effectiveness of different campaign messages or candidate attributes. Different versions of the survey may present respondents with varying descriptions of candidates’ qualifications, issue positions, or personal backgrounds. When you compare the responses across different groups, you can gain insights into the effects of these variations on voter preferences and decision-making.
• Product testing: Split-ballot designs are frequently utilized in product testing to assess consumer preferences and choices. For instance, participants may be presented with different versions of product packaging, pricing options, or product descriptions. This will help you determine which variations have a significant impact on consumer behavior. You can also use this to make decisions about marketing strategies or product development.
• Social studies: In social studies, split-ballot designs are valuable for examining public opinion on controversial issues or policy options. Different versions of the survey may present respondents with different arguments or policy proposals. Compare the responses to gain insights into how variations in framing or presentation affect public attitudes and preferences. This will also inform your policy development and decision-making.

These examples of split-ballot designs have provided valuable outcomes and insights. Researchers have identified specific messages, attributes, or factors that significantly influence voter preferences, consumer choices, or public opinion. 

Such findings have informed political campaign strategies, marketing campaigns, and policy development, enabling decision-makers to tailor their approaches and communications to better align with target audiences.

Challenges and Limitations of Split-Ballot Design

While split-ballot designs offer numerous benefits, there are several challenges and limitations to consider:

• Participant fatigue: Respondents may experience fatigue or frustration when exposed to lengthy or repetitive surveys, potentially affecting their responses. To mitigate this, researchers should carefully design surveys that are concise and engaging, reducing the risk of participant fatigue.
• Response bias: Even with random assignment, there is still a possibility of response bias due to individual differences or uncontrolled factors. You should be cautious in interpreting and generalizing the results, considering the potential for bias introduced by participants’ characteristics or attitudes.
• Potential confounding factors: Split-ballot designs aim to isolate the effects of specific variations; however, other factors may confound the results. You should carefully account for potential confounding variables and consider additional statistical techniques, such as covariate adjustment or matching, to control for these factors.
• Interpretation and consideration of results: Careful interpretation of the results is essential, considering the context and limitations of the split-ballot design. Researchers should acknowledge the potential impact of variations and provide a balanced interpretation of the findings, recognizing the complexity of human behavior and the multiple factors that influence responses. 

In conclusion, split-ballot design is a valuable technique in survey research that allows researchers to explore the impact of variations in question formats, stimuli, or other experimental conditions on participant responses. Researchers must carefully plan and manage the data collection process to ensure accuracy, reliability, and proper allocation of participants to each version.

When you employ this design, you can reduce bias, increase validity, and gain insights into the factors that influence survey results. While split-ballot designs offer numerous advantages, careful planning, and implementation are necessary to address potential challenges and ensure accurate data collection and analysis.

Connect to Formplus, Get Started Now - It's Free!

• split-ballot design
• survey bias
• survey design
• survey fatigue
• survey research
• survey research methods

You may also like:

Survey Straightlining: Definition, Implications & Mitigation

Introduction Survey straight lining occurs when the respondents of a survey in haste, select the same response every time....

What is Survey Panel Attrition?

Introduction Survey panel attrition is a critical aspect of survey research that deserves attention and understanding. It refers to the...

Monadic Vs Sequential Survey Design

Introduction Are you looking to test a new product idea or a service? They are quite a number of methods available in the market...

Paired Comparison Scale in Surveys: Purpose, Implementation, & Analysis

Introduction In survey research, capturing preferences and relative rankings is often a crucial objective. One effective tool for...

Formplus - For Seamless Data Collection

Collect data the right way with a versatile data collection tool. try formplus and transform your work productivity today..

• Search Menu

Sign in through your institution

• Author Guidelines
• Submission Site
• Open Access Options
• Why Submit?
• About International Journal of Public Opinion Research
• About the World Association for Public Opinion Research
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Journals on Oxford Academic
• Books on Oxford Academic

Lasswell’s Problem and Hovland’s Dilemma: Split-Ballot Experiments on the Effects of Potentially Emotionalizing Visual Elements in Media Reports

Thomas Petersen (PhD University of Mainz; postdoctoral lecture qualification University of Dresden) is a project director at the Allensabch Institute and an associate lecturer at the Universities of Mainz and Krems.

• Article contents
• Figures & tables
• Supplementary Data

Thomas Petersen, Lasswell’s Problem and Hovland’s Dilemma: Split-Ballot Experiments on the Effects of Potentially Emotionalizing Visual Elements in Media Reports, International Journal of Public Opinion Research , Volume 23, Issue 3, September 2011, Pages 251–264, https://doi.org/10.1093/ijpor/edq051

• Permissions Icon Permissions

The question of how the content of images that accompany reports in the mass media can be systematically assessed via content analysis and how the effects of such images can be quantitatively ascertained and related to textual reporting, is one of the still unsolved issues in empirical communication research. In this article, the results of two experiments are presented, in which the emotional reactions to manipulated news items was tested with the result that no significant effect in terms of emotionalization could be proven. Since other comparable experiments, where the text has been manipulated, have shown significant effects, the study seems to indicate that texts may have a stronger emotional effect than pictures.

World Association for Public Opinion Research members

Personal account.

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code
• Add your ORCID iD

Institutional access

Sign in with a library card.

• Sign in with username/password
• Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Short-term Access

To purchase short-term access, please sign in to your personal account above.

Don't already have a personal account? Register

Email alerts

Citing articles via.

• Recommend to Your Librarian

Affiliations

• Online ISSN 1471-6909
• Copyright © 2024 World Association for Public Opinion Research
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Our systems are now restored following recent technical disruption, and we’re working hard to catch up on publishing. We apologise for the inconvenience caused. Find out more: https://www.cambridge.org/universitypress/about-us/news-and-blogs/cambridge-university-press-publishing-update-following-technical-disruption

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings .

Login Alert

• > Experiments in Public Management Research
• > Survey Experiments for Public Management Research

Book contents

• Experiments in Public Management Research
• Copyright page
• Contributors
• Preface and Acknowledgements
• Part I Context
• Part II Methods
• 4 Causal Inference and the Design and Analysis of Experiments
• 5 Field Experiments in Public Management
• 6 Survey Experiments for Public Management Research
• 7 Laboratory Experiments: Their Potential for Public Management Research
• Part III Substantive Contributions
• Part IV Issues and Implications
• Appendix Recommended Reporting Guidelines for Experiments in Public Management, a Checklist1

6 - Survey Experiments for Public Management Research

from Part II - Methods

Published online by Cambridge University Press:  27 July 2017

Access options

Save book to kindle.

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle .

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service .

• Survey Experiments for Public Management Research
• By Sebastian Jilke , Gregg G. Van Ryzin
• Edited by Oliver James , University of Exeter , Sebastian R. Jilke , Rutgers University, New Jersey , Gregg G. Van Ryzin , Rutgers University, New Jersey
• Book: Experiments in Public Management Research
• Online publication: 27 July 2017
• Chapter DOI: https://doi.org/10.1017/9781316676912.007

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox .

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive .

Enter the URL below into your favorite RSS reader.

Using a split-ballot design to validate an abbreviated categorical measurement scale: An illustration using the Transportation Security Index

• Citation (BibTeX)

Sorry, something went wrong. Please try again.

If this problem reoccurs, please contact Scholastica Support

Error message:

View more stats

To address the high survey costs and increased respondent burden that comes with administering composite multi-item scales, researchers frequently seek to develop and use abbreviated scales. To help them do so, methodologists have issued a series of guidelines outlining best practices for shortening scales. However, it is difficult to find an empirical illustration of both the design and validation of an abbreviated scale, particularly one for which the classification of respondents into distinct categories is of paramount importance. In this paper, we present such an illustration using the Transportation Security Index (TSI) as a motivating example. Notably, we employ a split-ballot experiment to validate the TSI-6, a six-item abbreviated scale that successfully reproduces the original, validated TSI-16. We also illustrate the implementation of several agreed upon best practices in abbreviated scale development and propose and demonstrate specific steps that are uniquely relevant to the validation of a categorical abbreviated measure.

Introduction

The use of composite multi-item scales to measure latent (i.e., unobservable) constructs is widespread in survey research across the disciplines. Yet, the length of these scales (many upwards of 15 items) poses challenges for survey administration: high survey costs, increased respondent burden, and item non-response (Coste et al. 1997; Stanton et al. 2002; Smith , Combs , and Pearson 2012) . To address these challenges, researchers seek to define and use abbreviated scales (see, for example, Blumberg et al. 1999; Levine 2013 ).

Although shortening multi-item scales is common practice, as Goetz et al. (2013) point out, the strategies scholars use in the shortening process often lack “methodological rigor,” calling the validity of these abbreviated measures into question (p. 711). To address this, over the years, researchers have issued a series of methodological guidelines suggesting best practices for scale shortening (see, for example, Coste et al. 1997; Smith , McCarthy , and Anderson 2000 ; and Stanton et al. 2002 as cited in Goetz et al. 2013 ).

The emphasis of such guidelines is typically focused on the first phase of the shortening process: defining an abbreviated scale. Much less attention is given to the second phase of the shortening process: validation. To the extent that it is given attention, researchers widely recommend that the abbreviated scale be validated on an independent sample (Coste et al. 1997; Smith , McCarthy , and Anderson 2000; Stanton et al. 2002; Smith , Combs , and Pearson 2012; Goetz et al. 2013; Kruyen , Emons , and Sijtsma 2013; Sitarenios 2022) . Yet, the guidance stops short of recommending the use of split-ballot experiments, the gold-standard technique in evaluating question wording differences (Schuman and Presser 1981) . Further, little guidance outlines how to validate a categorical abbreviated scale (but see Smith , McCarthy , and Anderson 2000 ) and no guidelines, to our knowledge, provide empirical illustrations of successful validation exercises.

Therefore, in this paper, we provide a step-by-step empirical illustration of scale shortening that includes both phases of the shortening process. We begin by illustrating how to define a shortened version of a scale by following several agreed-upon recommended practices as described in the literature. We then illustrate how a split-ballot experiment can be used to validate an abbreviated categorical composite score. In doing so, this paper also provides an illustration of how to thoroughly document and justify decisions made throughout the process, a move widely recommended across the guidelines (see, for example, Smith , McCarthy , and Anderson 2000; Goetz et al. 2013 ).

Our motivating example is the abbreviation of the Transportation Security Index (TSI) from a 16-item measure to a 6-item measure. The TSI is a validated measure of transportation insecurity, a condition in which an individual is unable to regularly move from place to place in a safe or timely manner due to an absence of resources necessary for transportation (Gould-Werth , Griffin , and Murphy 2018; Murphy , Gould-Werth , and Griffin 2021) . Modeled after the Food Security Index (National Research Council 2006) , the TSI was designed to measure transportation insecurity at the individual level based on the way people experience it qualitatively, regardless of mode of transit or geography. The TSI has been cited as a useful measure of transportation-related material hardship (Murphy et al. 2022) , a valuable evaluation tool (Sung et al. 2023) , and a potentially useful screening tool for clinicians (Brandt et al. 2023) . Yet, as Turner et al. (2020) have pointed out, its 16-item length is burdensome and cost prohibitive for inclusion on most questionnaires, warranting the development of an abbreviated form.

To identify and validate an abbreviated TSI, we drew upon data derived from surveys and cognitive interviews.

Survey data. Survey data were gathered from two similar data collections administered in May 2018 and November 2022. The 2018 survey was fielded to validate the original TSI-16 (see Murphy , Gould-Werth , and Griffin 2021 ) and to develop a preliminary abbreviated scale. Accordingly, all respondents (analytic sample size = 1,999) were administered the full TSI-16. The 2022 survey was fielded to validate the proposed abbreviated scale and included a split-ballot experiment wherein one random half-sample (analytic sample size = 1,099) received the original TSI-16 and the other random half-sample (analytic sample size = 1,118) received the proposed abbreviated scale (See Appendix A for the 2022 survey questionnaire; items comprising the abbreviated scale are in bold font). Each survey was administered to a distinct sample of Ipsos’ (formerly GfK Group) KnowledgePanel® members. Recognizing that the unique transportation behaviors of college-aged young adults might impact our results, we restricted both survey samples to U.S. adults aged 25 years or older. Both surveys also included oversamples of respondents living in households at or below the federal poverty line. For further details about each of our data collection efforts, including information about the KnowledgePanel® and descriptive statistics of each sample, please refer to Appendix B.

Cognitive interview data. In 2015, to identify the initial pool of candidate TSI items, we conducted 52 cognitive interviews with a socioeconomically and demographically diverse group of respondents in Chicago and urban, suburban, and rural Michigan (see Gould-Werth , Griffin , and Murphy 2018 ). These cognitive interviews were again considered here. Respondents were identified through nonprofit organizations, door knocking, and snowball sampling. During the interview, respondents were administered our candidate items, probed to assess comprehension, recall, and judgement, and asked about their financial and transportation situations.

Methods & Results

In this section, we provide a step-by-step illustration of how to define and validate an abbreviated version of a categorical composite scale, using the TSI as an example. For each step, we begin by providing a methodological justification for the step, noting the recommended guidelines where they exist. We then detail how, for each step, we implemented these practices in the shortening of the TSI. Throughout this discussion, all survey data were weighted and analyzed using either Stata 15.1 (StataCorp 2017) or M plus 6.1 (Muthén and Muthén 1998–2010) .

Defining a shortened version of a scale

Step 1: Document the validity and measurement properties of the original scale. The methodological guidelines for shortening scales broadly agree that only those original scales that have been validated and demonstrated to have good measurement properties should be shortened (see, for example, Coste et al. 1997; Smith , McCarthy , and Anderson 2000; Goetz et al. 2013 ). [1] Because the abbreviated scale should preserve (or improve upon) the original scale’s psychometric properties, it is important to first document the psychometric properties (e.g., dimensionality, validity, reliability) of the original scale from which the abbreviated scale will be derived. As Goetz et al. (2013) argue, doing so enables potential users of the abbreviated scale to better understand how decisions around shortening were made.

The development of the TSI-16 was the result of a multi-step process. As described in Gould-Werth , Griffin , and Murphy (2018) , item content was informed by extensive qualitative research, including 187 interviews. A preliminary index was identified using exploratory factor analysis on survey data collected in 2016 (Gould-Werth , Griffin , and Murphy 2018) . This index was then validated on a different nationally representative survey sample (administered in 2018) by using confirmatory factor analysis and other analytic methods (Murphy , Gould-Werth , and Griffin 2021) . Used as a categorical measure, the TSI-16 identifies five categories of transportation insecurity generated from an individual’s sum score (0-2 = secure, 3-5 = marginal, 6-10 = low, 11-16 = moderate, 17-32 = high insecurity) (McDonald-Lopez et al. 2023) .

Step 2: Define an objective for the abbreviated scale. Methodological guidelines widely recommend that the objectives for defining an abbreviated scale be made explicit at the outset of the shortening process, and that they include the anticipated benefits to be derived from an abbreviated scale as well as how many items will be needed for this shortened scale to meet these goals (see, for example, Smith , McCarthy , and Anderson 2000; Goetz et al. 2013 ). Documenting such information is important not only because the defined objectives shape item selection and other methodological considerations, but also because, as Goetz et al. (2013) write, providing such information will help potential users of an index decide whether the original or shortened version of a scale should be administered.

With this in mind, we defined four objectives for our abbreviated TSI. First, taking our conceptual model into account as recommended by Goetz et al. (2013) , we wanted the abbreviated scale to efficiently capture both the material and relational manifestations of transportation insecurity (content validity) most likely to be encountered across a variety of survey contexts, including those with relatively smaller sample sizes. Second, we wanted the abbreviated scale to have face validity among both respondents and researchers. Face validity for respondents would increase respondent motivation and thus the quality of data collected. Face validity for researchers would facilitate the use of the scale in research. Third, we desired a categorical abbreviated scale that would demonstrate concordance with the type of transportation insecurity categories defined by the categorical original scale. Finally, given that empirical work using the TSI has focused on quantifying the prevalence of transportation insecurity (Murphy et al. 2022) , we aimed to develop an abbreviated TSI that would capture transportation security’s prevalence as precisely as the original scale does. Recognizing the generally low prevalence of the most severe categories of transportation insecurity (e.g., 3% and 5% of U.S. adults were estimated to experience high and moderate transportation insecurity, respectively [ Murphy et al. 2022 ]) and the likelihood of the measure being dichotomized in external analyses, we privileged items that distinguished between respondents experiencing transportation security and respondents experiencing any level of insecurity.

We did not identify a specific target length that would be needed to meet these objectives. We did, however, desire to identify a scale that had no fewer than three items, the minimum number of items required for a one-factor model.

Step 3: Use both content and statistical approaches to select items and document the item selection process. Detail the justification for item retention or removal, including whatever tradeoffs were made in such decisions. The literature suggests that it is a best practice to ensure that the abbreviated scale retains the psychometric properties of the original by using statistical approaches to evaluate what items should be retained or struck (see, for example, Coste et al. 1997; Smith , McCarthy , and Anderson 2000; Stanton et al. 2002; Goetz et al. 2013; Sitarenios 2022 ). Because it is also important to preserve the content validity of the original scale, methodological guidelines also widely recommend simultaneously taking the content of each item into account when conducting such an evaluation (see, for example, Coste et al. 1997; Stanton et al. 2002; Smith , McCarthy , and Anderson 2000; Goetz et al. 2013 ).

Following this logic, we approached shortening the TSI by considering what individual items we could justifiably discard. Evaluating the psychometric properties of each item (“statistical approach”), we began by ranking all 16 items by their item discrimination and item difficulty parameters (“never to sometimes”) as estimated by a graded response model using our 2018 survey data (see Table 2 ). Graded response models estimate the probability that a respondent will endorse a particular item response given the respondent’s location on a latent continuum (here, transportation insecurity), the ability of the item to differentiate among respondents at different locations on the latent continuum (item discrimination), and the location on the latent continuum at which the respondent has a 50 percent chance of endorsing a particular item response (item location). A desirable set of items will have high discrimination values while adequately covering the content space (i.e., including easier and more difficult items) (DeVellis 2017; Sitarenios 2022) .

Note : SE = standard error; final TSI-6 in bold font a Items sorted in decreasing order of item discrimination

Recognizing that individuals experiencing the greatest level of transportation are less likely to be detected in applications with smaller sample sizes, we first removed the most difficult item to endorse ( avoiding ). Next, although paying what Lowe and Mosby (2016) call the “time tax” is central to the experience of transportation insecurity, our results showed that the four items related to time ( late, took longer, early, waiting) were the least discriminating, likely because transportation secure people also perceive themselves to incur travel time costs (McDonald-Lopez et al. 2023) . Although the recommended guidelines for shortening scales emphasize the importance of preserving the content validity of the original scale, such considerations must be weighed against the fact that any abbreviated scale must only retain items that most efficiently differentiate those experiencing transportation insecurity from those who are transportation secure. Because these items do not accomplish this objective and because we are retaining other items that tap into the material dimension of insecurity, we elected to remove them.

Given that our statistical approach did not suggest striking any additional items, we drew on our cognitive interview data to evaluate the performance of each of our remaining 11 items (“content approach”). Analysis revealed that when thinking about feeling bad , respondents considered feelings related to feeling left out and embarrassed . Because feeling bad encompassed the two items that respondents interpreted more narrowly, thus producing semantic redundancy, we struck left out and embarrassed (see Stanton et al. 2002 for a discussion of eliminating items based on semantic redundancy). Similarly, we removed not invited , keeping the more general and all-encompassing relationship effects.

We decided to retain not able to leave house when you want to over stuck – items capturing a similar experience – for two reasons. First, admitting to “feeling stuck at home” might be perceived as stigmatizing by some respondents, potentially resulting in their disengagement from the response task. Such an item would thus undermine our objective of identifying an abbreviated scale that would increase respondent motivation. Second, in addition to capturing people who are stuck at home, not able to leave the house when you want to also captures the lack of autonomy that transportation insecure people experience when they have to rely on the schedules and reliability of public transit and social networks for rides and thus covers more symptoms associated with transportation insecurity.

Although “worry” questions have worked well in indices measuring other forms of material hardship, like food insecurity, our evaluation of respondent comprehension indicated that, in some cases, respondents interpreted worry overly broadly, to include, for example, concerns about inconveniences related to traffic or road construction. For this reason, we struck worry .

Ultimately, then, six items – 3 material and 3 relational – were retained for the abbreviated TSI, preserving the content validity of the original scale: reschedule , skipped , not able to leave house when you want to , felt bad , inconvenience, and relationship effects (see Table 2 ; TSI-6 items are in bold font).

Validating the abbreviated version of a scale

The validation of the abbreviated scale helps determine the extent to which the abbreviated scale preserves (or improves upon) the psychometric properties of the original scale, a necessary requirement of an effective abbreviated scale (Coste et al. 1997; Smith , McCarthy , and Anderson 2000; Goetz et al. 2013; Kruyen , Emons , and Sijtsma 2013; Sitarenios 2022) . Below, we describe the way we structured each step of the validation process, from data collection to analysis, in an effort to ensure a rigorous comparison of our abbreviated and original scales, thus demonstrating how to provide a convincing validation of an abbreviated scale.

Step 1: Conduct a split-ballot experiment on an independent sample using the same data collection procedures and sample design used in validating the original scale. To decrease the likelihood that the abbreviated scale would be overfitted to a particular sample, the literature recommends testing abbreviated indices on new, independent samples representing the same target population (see, for example, Coste et al. 1997; Smith , McCarthy , and Anderson 2000; Stanton et al. 2002; Smith , Combs , and Pearson 2012; Goetz et al. 2013; Kruyen , Emons , and Sijtsma 2013; Sitarenios 2022 ). More specifically, we recommend a split-ballot survey design wherein the original scale is administered to one random half-sample and the abbreviated scale to the other random half-sample. Such a technique is used widely to compare the effectiveness of question wording alternatives (see Schuman and Presser 1981 ) and is well suited for comparing different versions of measurement scales because it ensures that comparisons between the original and abbreviated scales are not conflated with any difference in the survey sample or data collection procedures. A split-ballot design also protects against “halo effects” which occur when only the original scale is administered and abbreviated items are extracted from it or when both the original and abbreviated forms are administered to the same sample in the same survey, two common practices in the literature (Goetz et al. 2013) . In such designs, responses to the abbreviated scale are likely influenced by the concurrent administration of the remaining original scale items, thus impacting the generalizability of the results.

Accordingly, in 2022, we fielded a new survey on an independent sample. We administered the original TSI-16 to one random half-sample (“Ballot One”) and the abbreviated TSI-6 to the other random half-sample (“Ballot Two”). To minimize the variability in comparisons across survey efforts due to differences in survey methods, in 2022, we contracted with the same firm (Ipsos) and used the same panel (Knowledge Panel®) as we used in our 2018 survey. We also used the same sampling parameters (i.e., adults over age 25 and an oversample of those below the poverty line).

Step 2: Evaluate the consistency of the original scale over time. In order for the proposed abbreviated scale to accurately represent the original scale, it is important to first determine that the original scale performs as expected in the new independent sample.

Because the reproduction of prevalence estimates is one of the objectives for our abbreviated scale, in our case, we compared prevalence estimates derived from the TSI-16 in 2018 and 2022 (Ballot One only, by definition). As illustrated in Table 3 , prevalence estimates across the five categories of transportation insecurity did not meaningfully vary. Thus, 2018 and 2022 data are comparable and an abbreviated scale derived from the 2022 data that performs as well as the original scale measured in 2022 should, on its face, also represent the original scale validated in 2018.

Step 3: Evaluate the psychometric properties of the abbreviated scale. To evaluate whether the abbreviated scale preserves the original scale’s psychometric properties, consider examining the abbreviated scale’s dimensionality, reliability, and concurrent validity (which, for a categorical scale is assessed in steps 5 and 6) (Coste et al. 1997; Smith , McCarthy , and Anderson 2000; Stanton et al. 2002; Goetz et al. 2013; Sitarenios 2022) .

Step 4: Create cut points for the abbreviated scale using data from respondents in the new sample who were administered the abbreviated scale. To evaluate whether the abbreviated scale reproduces prevalence estimates derived from the categorical original scale, abbreviated scale categories, or cut points, first need to be identified. This can be achieved using similar methods as were used in creating cut points for the original scale.

In our case, we conducted a k -means cluster analysis using data from Ballot Two (abbreviated scale only) respondents. In this non-deterministic partitional clustering method, observations are iteratively clustered into k mutually exclusive and exhaustive categories using their continuous TSI sum scores as input (MacQueen 1967) . Generally, smaller values of k will result in solutions that are more reproducible; however, meaningful substantive differences between observations might be missed. Therefore, we desired to identify a k which provided as much description of the population as could be generally reproduced. Given our prior identification of a five-category TSI-16 (secure, marginal, low, moderate, high insecurity), we determined that between three and five distinct categories of transportation insecurity might be identified using the abbreviated scale. Accordingly, we estimated k =3, k =4, and k =5 means clustering models. Because the method is nondeterministic (i.e., results could differ each time the model is estimated), we re-estimated each model 10 times.

As illustrated in Table 4 , among the 3-, 4-, and 5-cluster solutions estimated, only the 3-cluster solution exhibited consistent replication across a majority of iterations. This solution identified three clusters defined by sum scores of 0–1 (secure), 2–5 (marginal/low), and 6–12 (moderate/high). These clusters thus define our preliminary three-category abbreviated scale.

*For ease of interpretation, cluster solutions have been rearranged so that identical solutions are adjacent.

Step 5: Calculate the level of agreement between the original and abbreviated scales. Such calculations (e.g., percent agreement, Kappa statistic) determine the extent to which the categorical abbreviated scale aligns with the categorical original scale (i.e., concurrent validity) (Coste et al. 1997; Smith , McCarthy , and Anderson 2000; Stanton et al. 2002; Goetz et al. 2013) .

In our case, because the number of categories between scales differed, we began by examining the distribution of the five TSI-16 original scale categories across the continuous TSI-6 abbreviated scale sum scores among only Ballot One respondents. As expected, the percentage of respondents classified as “secure” (value 1) (per the original scale) decreases as the abbreviated scale sum score increases (see Figure 1 ). Furthermore, the pattern suggests that the three categories identified in the abbreviated scale closely resemble a collapsed original scale categorization: The first categories of both the abbreviated and original scale generally identify respondents who are transportation secure. The second category of the abbreviated scale (sum scores between 2 and 5, inclusive) primarily identifies respondents who experience marginal or low insecurity (per the original scale; values 2 and 3). Finally, the third category of the abbreviated scale (sum scores between 6 and 12, inclusive) primarily identifies respondents who experience moderate or high insecurity (per the original scale; values 4 and 5).

To more formally estimate the concordance between the categorical original and abbreviated scales, we calculated the percent agreement between the two using the 2022 survey data. As illustrated in Table 5 , 90.8 percent (weighted) of all respondents completing Ballot One were similarly classified across both forms: 78.2 percent as transportation secure between scales, 6.7 percent as experiencing marginal or low insecurity, and 5.9 percent as experiencing moderate or high insecurity.

Because the simple percent agreement between two measures does not take into account chance agreement, we next estimated the Kappa statistic between the three-category abbreviated scale and the three-category original scale that was created by collapsing the original five categories as discussed above (i.e., 1=1, 2=2,3, 3=4,5). As estimated on the Ballot One sample, the Kappa statistic was 0.76, reflecting substantial (Landis and Koch 1977) or excellent (Fleiss , Levin , and Paik 1981) agreement.

Step 6: Use chi-square analysis to compare prevalence estimates derived from the original and abbreviated scales. Because the performance of an abbreviated categorical scale depends on its ability to classify people in the same way the original scale does (Smith , McCarthy , and Anderson 2000; Smith , Combs , and Pearson 2012; Kemper et al. 2018; Sitarenios 2022) , prevalence estimates derived from the original and abbreviated scales should be compared. To do this, create a single x -category variable across the entire new data set such that respondents who received the ballot with the original scale are assigned their x -category original scale score and respondents who received the abbreviated scale are assigned their x -category abbreviated scale score. Next, to determine whether there is a significant difference in prevalence estimates between the two scales, conduct a chi-square analysis.

Toward that end, we created a single three-category TSI variable across the entire 2022 data set such that Ballot One respondents were assigned to one of three categories defined by the original scale score cut points, and Ballot Two respondents were assigned to one of three categories defined by the abbreviated scale score cut points. We then conducted a weighted chi-square analysis which revealed no significant difference in prevalence estimates between the two scales (see Table 6 ; design-based F(1.99, 4406.47)=1.7910, p =0.167). There is initial evidence, then, that the TSI-6 is a sufficient proxy for the TSI-16 when estimating transportation insecurity’s prevalence.

Design-based F(1.99, 4406.47) = 1.7910, p =0.167

This paper presented the steps we took to define and validate the TSI-6. By doing so, we aimed to provide readers with a useful empirical illustration of how to define and validate an abbreviated categorical scale in line with some of the best practices in survey research. It is our hope that such an illustration also provides a useful example of how to thoroughly document and justify all decisions and considerations made throughout the shortening process. As the methodological guidelines recommending such transparency note, providing such documentation is important because it provides potential users of the abbreviated measure with the information they need to evaluate its strengths and weaknesses and whether they wish to use it (Smith , McCarthy , and Anderson 2000; Goetz et al. 2013) .

Our example was the shortening of the 16-item Transportation Security Index (TSI-16). Using nationally representative survey data and cognitive interview data and drawing upon statistical and content approaches, we developed and validated the TSI-6: six questions that can be used to determine one’s level of transportation insecurity. Importantly, this abbreviated scale met our objectives as outlined at the beginning of the paper: (1) the scale captures both the material and relational manifestations of transportation insecurity, (2) the items have face validity, (3) the scale identifies comparable categories of insecurity as the original scale, and (4) the scale generates comparable prevalence estimates as the original scale. Therefore, the TSI-6 can be used to achieve parsimony with little loss of information. Moreover, it can do so while decreasing respondent burden and survey costs. Based on our 2022 survey data, whereas the median time to complete the TSI-16 was 2.12 minutes, the median time to complete the TSI-6 was just under 1 minute.

Importantly, the specific processes we followed were iterative and dependent on the unique properties of our scale, our research objectives, and the results each step garnered. For example, our example involved shortening a unidimensional scale. There are many composite scales, however, that have multiple factors, each of which must be preserved in the shortening process (Smith , McCarthy , and Anderson 2000; Goetz et al. 2013) . Our example also involved validating a single defined abbreviated scale. There are other cases, however, where researchers might be considering multiple abbreviated scale options. In these cases, we would recommend an experiment including one ballot for the original scale and one ballot for each of the proposed abbreviated scales. Finally, because we aimed to develop an abbreviated scale that would capture transportation insecurity’s prevalence as precisely as the original scale does, our validation efforts placed special emphasis on comparing how the abbreviated scale performed against the original scale with respect to prevalence. Other researchers may have additional objectives for their abbreviated scales which should guide their validation efforts (Sitarenios 2022) . For instance, those who are interested in preserving the predictive validity of their original scale will want to add an additional step to their validation efforts: a comparison of how the defined abbreviate scale compares with the original scale in predicting some outcome of interest (Stanton et al. 2002) .

Depending on the broader research objectives of a study, using an abbreviated scale might not always be preferred to using the original scale. As is the case with many survey design decisions, the tradeoffs must be carefully weighed. For example, it might not be worth decreasing the overall survey length or reducing other survey costs, when the psychometric properties of the abbreviated scale are worse than those of the original scale (Kemper et al. 2018) . Furthermore, as in our example presented here, the abbreviated scale might identify a coarser categorization of the latent construct than does the original scale. To the extent that greater differentiation of respondents is desired, questionnaire space is available, and survey sample sizes are sufficient, the original scale may be the preferred measure.

Of course, measurement development is an ongoing process. As with the development and validation of original scales, once an abbreviated scale has been validated, researchers should seek to replicate their findings in different survey contexts, examining how the abbreviated scale performs with different modes of administration, target populations, and questionnaire contexts.

Lead Author

Alexandra K. Murphy Department of Sociology University of Michigan 3115 LSA Building 500 South State Street Ann Arbor, MI 48109

Acknowledgements

We thank Mike Bader and David Pedulla for providing feedback during the early stages of our work defining the abbreviated TSI. We are also grateful to the following agencies whose financial support made this publication possible: National Science Foundation (grant OIA09936884); the Stanford Center on Poverty and Inequality (grant H79AE000101 from the US Department of Health and Human Services); and the University of Michigan’s Poverty Solutions and Mcity initiatives, College of Literature, Science, and the Arts, Office of Research, and Department of Sociology. Any opinions, findings, and conclusions or recommendations expressed in this article are those of the author(s) and do not necessarily reflect the views or official policies of the National Science Foundation or the US Department of Health and Human Services.

Submitted : August 18, 2023 EDT

Accepted : November 04, 2023 EDT

Appendix A: Transportation Security Index 2022 Survey Questionnaire

Note to Reader: Bold font is used to identify the six items that comprise the TSI-6. Importantly, for Q9 that asks, “In the past 30 days, how often were you not able to leave the house when you wanted to because of a problem with transportation?” the question is presented to respondents as it appears here, with the word “ not ” in bold font. [S] denotes items where only one response was allowed. [M] denotes items where multiple responses were allowed. Question 1 technically consists of several questions that gather updated information about household size and household income. No question number was assigned to these questions, however.

[DISP_INTRO]

Before we begin the survey, we’d like to ask you some questions about your household. Please keep in mind that your answers are confidential and your personal information will also be kept private. We appreciate your participation in this important study!

Base: All respondents

QHHSIZE_adults [Q]

Including yourself, how many people are 18 years of age or older and currently live in your household at least 50% of the time?

Please include unrelated individuals (such as roommates), and also include those now away traveling, away at school, or in a hospital.

Your answer will help represent the entire U.S. population and will be kept confidential. Thank you!

Type in the number of adults 18 years of age or older.

SCRIPTER: min.=1, max.=10. Prompt following nonresponse. Show on same screen as Q5b.

QHHSIZE_kids [Q]

Next, how many people are 17 years of age or younger and currently live in your household at least 50% of the time? If none, enter “0”.

Include babies and small children.

Type in the number of children 17 years of age or younger.

SCRIPTER: min.=0, max.=10. Prompt following nonresponse.

QHHSIZE [Q]

SCRIPTER: Create DOV: QHHSIZE=QHHSIZE_adults + QHHSIZE_kids. Compute if QHHSIZE_adults and QHHSIZE_kids are not refused.

Base: all respondents

How much is the combined income of all members of YOUR HOUSEHOLD for the PAST 12 MONTHS?

Please include your income PLUS the income of all members living in your household (including cohabiting partners and armed forces members living at home). Please count income BEFORE TAXES and from all sources (such as wages, salaries, tips, net income from a business, interest, dividends, child support, alimony, and Social Security, public assistance, pensions, or retirement benefits).

Select one answer only.

Below $50,000

$50,000 or more

SCRIPTER: Prompt once if question is skipped. Do not show ‘Don’t know’ initially. Show ‘Don’t know’ only with the prompt if question is skipped initially.

Base: respondents with household income below $50,000 (QINC=1)

We would like to get a better estimate of your total HOUSEHOLD income in the past 12 months before taxes. Was it…

Less than $5,000

$5,000 to $7,499

$7,500 to $9,999

$10,000 to $12,499

$12,500 to $14,999

$15,000 to $19,999

$20,000 to $24,999

$25,000 to $29,999

$30,000 to $34,999

$35,000 to $39,999

$40,000 to $49,999

Base: respondents with household income of $50,000 or more (QINC=2)

$50,000 to $59,999

$60,000 to $74,999

$75,000 to $84,999

$85,000 to $99,999

$100,000 to $124,999

$125,000 to $149,999

$150,000 to $174,999

$175,000 to $199,999

$200,000 to $249,999

$250,000 or more

SCRIPTER: Create Data-only variables below.

Variable name: PPINCIMP [S]

Variable Text: HH income

Response list

if pphhsize = 1 and ppincimp le 4 FPL100 = 1.

if pphhsize = 2 and ppincimp le 5 FPL100 = 1.

if pphhsize = 3 and ppincimp le 6 FPL100 = 1.

if pphhsize = 4 and ppincimp le 7 FPL100 = 1.

if pphhsize = 5 and ppincimp le 8 FPL100 = 1.

if pphhsize = 6 and ppincimp le 9 FPL100 = 1.

if pphhsize = 7 and ppincimp le 10 FPL100 = 1.

if pphhsize = 8 and ppincimp le 10 FPL100 = 1.

if pphhsize = 9 and ppincimp le 11 FPL100 = 1.

if pphhsize = 10 and ppincimp le 11 FPL100 = 1.

if pphhsize = 11 and ppincimp le 12 FPL100 = 1.

if pphhsize = 12 and ppincimp le 12 FPL100 = 1.

if pphhsize = 13 and ppincimp le 12 FPL100 = 1.

if pphhsize = 14 and ppincimp le 13 FPL100 = 1.

if pphhsize = 15 and ppincimp le 13 FPL100 = 1.

if pphhsize = 16 and ppincimp le 13 FPL100 = 1.

if pphhsize = 1 and ppstaten = 94 and ppincimp le 5 FPL100 = 1.

if pphhsize = 2 and ppstaten = 94 and ppincimp le 6 FPL100 = 1.

if pphhsize = 3 and ppstaten = 94 and ppincimp le 7 FPL100 = 1.

if pphhsize = 4 and ppstaten = 94 and ppincimp le 9 FPL100 = 1.

if pphhsize = 5 and ppstaten = 94 and ppincimp le 10 FPL100 = 1.

if pphhsize = 6 and ppstaten = 94 and ppincimp le 10 FPL100 = 1.

if pphhsize = 7 and ppstaten = 94 and ppincimp le 11 FPL100 = 1.

if pphhsize = 8 and ppstaten = 94 and ppincimp le 12 FPL100 = 1.

if pphhsize = 9 and ppstaten = 94 and ppincimp le 12 FPL100 = 1.

if pphhsize = 10 and ppstaten = 94 and ppincimp le 12 FPL100 = 1.

if pphhsize = 11 and ppstaten = 94 and ppincimp le 13 FPL100 = 1.

if pphhsize = 12 and ppstaten = 94 and ppincimp le 13 FPL100 = 1.

if pphhsize = 13 and ppstaten = 94 and ppincimp le 14 FPL100 = 1.

if pphhsize = 14 and ppstaten = 94 and ppincimp le 14 FPL100 = 1.

if pphhsize = 15 and ppstaten = 94 and ppincimp le 15 FPL100 = 1.

if pphhsize = 16 and ppstaten = 94 and ppincimp le 15 FPL100 = 1.

if pphhsize = 1 and ppstaten = 95 and ppincimp le 5 FPL100 = 1.

if pphhsize = 2 and ppstaten = 95 and ppincimp le 6 FPL100 = 1.

if pphhsize = 3 and ppstaten = 95 and ppincimp le 7 FPL100 = 1.

if pphhsize = 4 and ppstaten = 95 and ppincimp le 8 FPL100 = 1.

if pphhsize = 5 and ppstaten = 95 and ppincimp le 9 FPL100 = 1.

if pphhsize = 6 and ppstaten = 95 and ppincimp le 10 FPL100 = 1.

if pphhsize = 7 and ppstaten = 95 and ppincimp le 11 FPL100 = 1.

if pphhsize = 8 and ppstaten = 95 and ppincimp le 11 FPL100 = 1.

if pphhsize = 9 and ppstaten = 95 and ppincimp le 12 FPL100 = 1.

if pphhsize = 10 and ppstaten = 95 and ppincimp le 12 FPL100 = 1.

if pphhsize = 11 and ppstaten = 95 and ppincimp le 12 FPL100 = 1.

if pphhsize = 12 and ppstaten = 95 and ppincimp le 13 FPL100 = 1.

if pphhsize = 13 and ppstaten = 95 and ppincimp le 13 FPL100 = 1.

if pphhsize = 14 and ppstaten = 95 and ppincimp le 14 FPL100 = 1.

if pphhsize = 15 and ppstaten = 95 and ppincimp le 14 FPL100 = 1.

if pphhsize = 16 and ppstaten = 95 and ppincimp le 14 FPL100 = 1.

All else, FPL100=0.

SCRIPTER: IF XRIDE=2 AND FPL100=0, TERMINATE AND INSERT STANDARD CLOSE.

Main survey

SCRIPTER: Split sample survey into two groups. Split sample xride=1 and 2 separately. Create DOV:

SPLIT_SAMPLE

1 = Ballot 1

2 = Ballot 2

Each question will be asked to all respondents Ballot 1 and 2 unless specified in base logic.

Base: all respondents (SPLIT_SAMPLE=1 and 2)

[DISPLAY 1]

Thank you for participating in this survey about how you get from place to place. The goal of this study is to understand people’s experiences with transportation and how these experiences shape their daily lives. We’ll start off by asking some questions about the focus of this survey: transportation.

Q2 [S per statement] [ACCORDION GRID]

How often do you use each of the following to get from place to place? If the type of transportation is not available to you, please select “Not available to me.”

Statements in rows:

Riding a motorcycle or moped

Your own personal vehicle (e.g., car, truck, SUV)

Borrowing the personal vehicle of a friend, family member, neighbor, coworker, or acquaintance

Getting a ride from a friend, family member, neighbor, coworker, or acquaintance (including carpooling)

Taking a taxi service or rideshare (e.g., Uber, Lyft)

Using a rental car or car sharing service (e.g., zipcar, Car2go)

Taking the bus

Taking the train or subway

Using paratransit (that is, specialized, door-to-door transport service for people with disabilities)

Statements in columns:

A few times a week

A few times a month

A few times a year

Not available to me

Base: ask if SPLIT_SAMPLE=1 (Ballot 1)

To get to the places they need to go, people might walk, bike, take a bus, train or taxi, drive a car, or get a ride. In the past 30 days, how often were you late getting somewhere because of a problem with transportation?

In the past 30 days, how often did it take you longer to get somewhere than it would have taken you if you had different transportation?

There are times when we need to wait for transportation to pick us up. In the past 30 days, how often did you spend a long time waiting because you did not have the transportation that would allow you to come and go when you wanted?

In the past 30 days, how often did you have to arrive somewhere early and wait because of the schedule of the bus, train, or person giving you a ride?

[If SPLIT_SAMPLE=2: To get to the places they need to go, people might walk, bike, take a bus, train or taxi, drive a car, or get a ride.] In the past 30 days, how often did you have to reschedule an appointment because of a problem with transportation?

In the past 30 days, how often did you skip going somewhere because of a problem with transportation?

In the past 30 days, how often were you not able to leave the house when you wanted to because of a problem with transportation?

In the past 30 days, how often did you worry about whether or not you would be able to get somewhere because of a problem with transportation?

In the past 30 days, how often did you feel stuck at home because of a problem with transportation?

In the past 30 days, how often do you think that someone did not invite you to something because of problems with transportation?

In the past 30 days, how often did you feel like friends, family, or neighbors were avoiding you because you needed help with transportation?

In the past 30 days, how often did you feel left out because you did not have the transportation you needed?

In the past 30 days, how often did you feel bad because you did not have the transportation you needed?

In the past 30 days, how often did you worry about inconveniencing your friends, family, or neighbors because you needed help with transportation?

In the past 30 days, how often did problems with transportation affect your relationships with others?

In the past 30 days, how often did you feel embarrassed because you did not have the transportation you needed?

Can you usually afford the transportation you need?

Q20 [S per statement] [BANKED GRID]

In the past 30 days, did you have trouble paying for any of the following?

Statements in row:

Car or vehicle payments

Vehicle insurance

Vehicle registration

Vehicle repairs

Outstanding traffic tickets (e.g., speeding, parking, driving without a license)

Paying a friend, family member, neighbor, coworker, or acquaintance for a ride

Taxi service or rideshare (e.g., Uber, Lyft)

Rental car or car sharing service (e.g., zipcar, Car2go)

Train or subway fare

Tolls or monthly toll passes

Statements in column:

Do you or does anyone else in your household own or lease a car or other vehicle for personal use?

Base: ask if Q21=1 or refused

Q22 [NUMBOX, 0-50]

Altogether, how many vehicles are owned, leased, or available for regular use by the people who currently live in your household? Please be sure to include motorcycles and mopeds.

__ __ Number of vehicles

Is the vehicle you use most of the time covered by insurance?

Do you currently have a valid driver’s license?

Transportation insecurity is a condition in which a person is unable to move from place to place in a safe or timely manner because they lack the financial or other resources necessary for transportation. In the past 30 days, how often have you experienced transportation insecurity?

Q26 [O] [PROMPT]

Please describe how you get from place to place and any problems you have with transportation.

[LARGE TEXTBOX]

Q27 [O] [PROMPT]

How, if at all, has your transportation situation changed since 2019?

Next, we would like to know a bit about your health and wellbeing.

In general, how would you rate your health?

Q29 [S per statement] [ACCORDION GRID]

Below is a list of ways you might have felt or behaved recently. How often have you felt or behaved in each of the following ways during the past week ?

I did not feel like eating; my appetite was poor.

I had trouble keeping my mind on what I was doing.

I felt depressed.

I felt that everything I did was an effort.

My sleep was restless.

I felt sad.

I could not get “going.”

Rarely or none of the time (less than 1 day)

Some or a little of the time (1-2 days)

Occasionally or a moderate amount of the time (3-4 days)

Most or all of the time (5-7 days)

The next questions are about whether you have difficulty with certain daily activities.

Q30 [S per statement] [BANKED GRID]

Statements in a row:

Do you have serious difficulty hearing?

Do you have serious difficulty seeing even when wearing glasses?

Do you have serious difficulty walking or climbing stairs?

Do you have difficulty dressing or bathing?

Statements in a column:

Q31 [S per statement] [BANKED GRID]

Because of a physical, mental, or emotional condition, do you have:

Serious difficulty concentrating, remembering, or making decisions?

Difficulty doing errands ALONE such as visiting a doctor’s office or shopping?

Now a question about what you do. Are you…?

Working now

Only temporarily laid off, or on sick or parental leave

Looking for work, unemployed

Permanently or temporarily disabled

Keeping house

Other (please specify) [O]

[DISPLAY 5]

We are interested in some of the problems people might face making ends meet. First, we are going to ask you about some of the bills you pay.

33 [S per statement] [ACCORDION GRID]

Thinking about your most recent bill, the one you paid in the past 30 days:

Did you pay the full amount of your rent or mortgage payment?

Did you pay the full amount of your water bill?

Did you pay the full amount of your gas, oil, or electric bill?

Did you pay the full amount of your phone or internet bill?

No, but I paid some

No, I skipped paying this bill

Not applicable/I don’t pay this bill

Q34 [S per statement] [BANKED GRID]

In the past 30 days, were any of the following services cut off because there wasn’t enough money?

Your gas/oil or electricity

Your phone or internet

[DISPLAY 6]

Now we are going to ask you about some other experiences you may have had in the last 30 days.

Q35 [S per statement] [ACCORDION GRID]

In the last 30 days, how often were each of the following statements true for you [if PPHHSIZE>1: or your household]?

The food that [if PPHHSIZE = 1: I; if PPHHSIZE > 1: we]/ bought just didn’t last, and [if PPHHSIZE = 1: I; if PPHHSIZE > 1: we]didn’t have money to get more.

[if PPHHSIZE = 1: I; if PPHHSIZE > 1: we] couldn’t afford to eat balanced meals.

Sometimes true

In the last 30 days, did you [if PPHHSIZE>1: or other adults in your household] ever cut the size of your meals or skip meals because there wasn’t enough money for food?

Base: ask if Q36=1 or refused

Q37 [NUMBOX, 0-30]

In the last 30 days, how many days did this happen?

______ days

Q38 [S per statement] [Banked grid]

In the last 30 days

Statements in a row

Did you ever eat less than you felt you should because there wasn’t enough money for food?

Were you ever hungry but didn’t eat because there wasn’t enough money for food?

Q39 [S per statement] [Banked grid]

In the past 30 days, did any of the following things happen to you or someone in your household?

Someone needed to see a doctor or go to the hospital but could not because of the cost.

Someone needed to get a prescription filled but could not because of the cost.

Someone needed to go to the dentist but could not because of the cost.

Q40 [S per statement] [Banked grid]

In the past 30 days, have any of the following things happened to you, even for one night?

You moved in with other people because of financial problems.

You stayed in a shelter.

You stayed in another place not meant for regular housing like an abandoned building or an automobile.

You were evicted or your landlord forced you to leave your home or apartment for not paying the rent or mortgage.

Q41 [S per statement] [Banked grid]

In the past 30 days, did you do any of the following to make ends meet?

Cut back on spending.

Sell something you own.

Take out a new loan from friends or family.

Take out a new loan from a private company (e.g., payday, title, bank).

Q42 [S per statement] [Banked grid]

In the past 30 days, did you receive any of the following?

Social Security (Old Age Social Insurance)

Disability benefits (SSI or SSD)

Unemployment benefits

Workers’ compensation

Food stamps (SNAP) or WIC (food benefits for women, infants, and children)

TANF (also called Temporary Assistance for Needy Families or cash assistance)

Housing assistance (includes rent vouchers and public housing)

Transportation assistance to help you get to work, school, training, or doctor’s appointments (includes gas vouchers, rideshare vouchers, bus passes, help repairing a car)

Other benefits (includes Life Line phones, childcare vouchers or other child care benefits, and LIHEAP assistance for heating and cooling costs)

Free food from a food bank

Assistance from a charity, church or some other organization

Base: ask if xppp20197=5 (missing)

Are you a citizen of the United States?

SCRIPTER: Prompt following nonresponse.

Base: ask if QEG22=1 or xppp20198=5 (missing)

Were you born a United States citizen or are you a naturalized U.S. citizen?

Born a U.S. citizen

Naturalized U.S. citizen

Appendix B: Additional Data Collection Details and Sample Characteristics

The KnowledgePanel® is an online panel survey administered to a sample representative of the non-institutionalized adult population of the United States, recruited using probability-based sampling and an address-based sample frame. If needed, respondents receive Internet access and a Web-enabled device. Analysis of KnowledgePanel® data aligns with benchmarks from data collected using gold-standard methods, such as U.S. Census data (Yeager et al. 2011) . Importantly for this study, the KnowledgePanel® sample frame has better coverage of minority racial and ethnic groups and low-income households than most random-digit-dial samples (Dennis 2010) .

As detailed in the paper, whereas all respondents to the 2018 survey were administered the full TSI-16, the 2022 survey included a split-ballot experiment such that one random half-sample (“Ballot One”) received the original scale TSI-16 and the other random half-sample (“Ballot Two”) received the proposed abbreviated scale TSI-6. As shown in Appendix Table 1 , certain completed cases were disqualified from the sample because they were initially selected for the oversample but did not actually meet the oversample eligibility criteria (that is, their household income was above the federal poverty line). Further, a small number of qualified respondents were dropped from each analytic sample because they did not complete any of the items that comprised the TSI version presented to them.

In both surveys, data were weighted to adjust for the complex survey design and unit nonresponse and post-stratification weights adjusted the sample to be representative of the U.S. population. See Appendix Table 2 for descriptive statistics of each sample.

1 Q35 (2018); Q21 (2022): Someone in household owns or leases car or other vehicle for personal use

But see Coste et al. (1997) who note that in cases where no gold-standard scale exists, researchers may seek to shorten existing scales in order to improve the measure’s psychometric properties. In these instances, the processes required to define and validate an abbreviated form differ from those described here. See Coste et al. (1997) for more details.

General Social Survey, 1972-2012 [Cumulative File] (ICPSR 34802)

Version Date: Sep 11, 2013 View help for published

Smith, Tom W., Hout, Michael, and Marsden, Peter V. General Social Survey, 1972-2012 [Cumulative File]. Roper Center for Public Opinion Research, University of Connecticut [distributor], Inter-university Consortium for Political and Social Research [distributor], 2013-09-11. https://doi.org/10.3886/ICPSR34802.v1

Export Citation:

• RIS (generic format for RefWorks, EndNote, etc.)

Principal Investigator(s): View help for Principal Investigator(s) Tom W. Smith , National Opinion Research Center; Michael Hout , National Opinion Research Center; Peter V. Marsden , National Opinion Research Center

• General Social Survey Series

https://doi.org/10.3886/ICPSR34802.v1

• At A Glance
• Export Metadata

Project Description

Summary view help for summary.

The General Social Surveys (GSS) were designed as part of a data diffusion project in 1972. The GSS replicated questionnaire items and wording in order to facilitate time-trend studies. The latest survey, GSS 2012, includes a cumulative file that merges all 29 General Social Surveys into a single file containing data from 1972 to 2012. The items appearing in the surveys are one of three types: Permanent questions that occur on each survey, rotating questions that appear on two out of every three surveys (1973, 1974, and 1976, or 1973, 1975, and 1976), and a few occasional questions such as split ballot experiments that occur in a single survey. The 2012 surveys included seven topic modules: Jewish identity, generosity, workplace violence, science, skin tone, and modules for experimental and miscellaneous questions. The International Social Survey Program (ISSP) module included in the 2012 survey was gender. The data also contain several variables describing the demographic characteristics of the respondents.

Citation View help for Citation

Funding view help for funding, subject terms view help for subject terms, geographic coverage view help for geographic coverage, smallest geographic unit view help for smallest geographic unit.

census region

Distributor(s) View help for Distributor(s)

Scope of project, time period(s) view help for time period(s), date of collection view help for date of collection, data collection notes view help for data collection notes.

Please note that NORC may have updated the General Social Survey data files. Additional information regarding the General Social Surveys can be found at the General Social Survey (GSS) Web site.

Methodology

Sample view help for sample.

For sampling information, please see Appendix A of the ICPSR Codebook.

Universe View help for Universe

All noninstitutionalized, English and Spanish speaking persons 18 years of age or older, living in the United States.

Unit(s) of Observation View help for Unit(s) of Observation

Data type(s) view help for data type(s), mode of data collection view help for mode of data collection, original release date view help for original release date, version history view help for version history.

• Smith, Tom W., Michael Hout, and Peter V. Marsden. General Social Survey, 1972-2012 [Cumulative File]. ICPSR34802-v1. Storrs, CT: Roper Center for Public Opinion Research, University of Connecticut/Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributors], 2013-09-11. http://doi.org/10.3886/ICPSR34802.v1

2013-09-11 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

• Created online analysis version with question text.
• Checked for undocumented or out-of-range codes.

Analysis Information

Weight view help for weight.

Due to the number of weights and various uses for them, users should refer to Appendix A of the ICPSR Codebook.

These data are freely available to data users at ICPSR member institutions . The curation and dissemination of this study are provided by the institutional members of ICPSR. How do I access ICPSR data if I am not at a member institution?