Harnessing AI for Designing Data Collection Instruments

Research Tips Blog

Published: Mar 14, 2025

In recent years, generative artificial intelligence (AI) has become a powerful tool for researchers in library and information science (LIS). Whether conducting user experience research, evaluating digital library services, or studying reading behaviors, LIS scholars often rely on well-designed data collection instruments such as surveys, interview guides, and codebooks for content analysis. AI can enhance these processes and increase efficiency in designing data collection instruments. Yet, AI-generated content should always be critically evaluated and revised to align with best practices in research methodology.

For example, in survey design, generative AI can assist in brainstorming measurable concepts, generating question ideas, and refining wording, but it does not replace our responsibility as the survey designer. If we are unsure how to break down our research topic into measurable components, we may use AI to generate ideas for survey constructs and key variables. An example of prompt would be “I am designing a survey to study how college students in entirely online programs perceive and use the university library. I hope to use the survey findings to help librarians better understand how to effectively provide services to this population. I need to identify measurable concepts that I should include in my survey. Please break down my research question into key variables and measurable concepts. Suggest specific survey topics that can effectively capture data on this topic. Also, highlight any potential biases or limitations I should be aware of when designing the survey”.

If we choose to use AI to draft questions, we must provide a clear and specific prompt that includes:

  • The research question and objectives.
  • The intended audience of the survey.
  • The type and number of questions required (e.g., open-ended, multiple-choice, Likert scales).
  • Any necessary constraints, such as language style or question length.

An example of prompt would be – “I am designing a survey to study how college students in entirely online programs perceive and use the university library. The survey needs to be 8-12 questions in length and cover the following concepts: Library Awareness & Access (Awareness of available library resources (digital collections, research assistance, interlibrary loan, frequency of library website visits or interactions with library services); Perceptions of Library Services (satisfaction with the accessibility and usability of online library resources, perceived usefulness of different library services, challenges faced in accessing library services remotely); Library Usage Behavior (types of library resources used most frequently; reasons for using the library; preferred modes of interaction with librarians); Barriers to Effective Use; and Student Preferences & Recommendations. The survey respondents are undergraduate students in a large urban university. The questions should be mostly closed questions. Only one open-ended is allowed at most. The wording of the questions needs to follow the basic rules of survey design, including but not limited to the following – make sure wording is concise, unambiguous, relevant, and specific; avoid double barreled questions; response categories are exhaustive and mutually exclusive; avoid leading and biased questions; be mindful about sensitive questions; sequence the survey questions properly; and word questions in ways to reduce recall bias and social desirability bias. Finally identify which survey questions correspond with which concept listed above.”

It’s important to note that AI-generated questions may contain biases, inconsistencies, or unclear wording—our job is to evaluate, refine, and ensure that each question meets survey design standards (e.g., clarity, neutrality, and relevance). Also, we need to be familiar with core survey design principles. This includes understanding question phrasing, avoiding leading or double-barreled questions, and structuring surveys for logical flow and reliability. AI outputs should always be reviewed and adjusted to align with the research goals and ethical considerations.

Similarly, when crafting interview guides for semi-structured interviews, AI can assist in generating interview questions tailored to research objectives; and when developing a codebook in coding qualitative data, AI can help by extracting common themes, generating initial coding schemes, and automating intercoder reliability checks.

Again, AI is not a replacement for human expertise but a valuable collaborator that can accelerate the research process, enhance objectivity, and reduce cognitive load. As AI continues to evolve, researchers must remain critical and ethical in its application, ensuring transparency, fairness, and interpretability in AI-assisted methodologies.

 

Comments

Post new comment