Useful contribution to discussions in this area, for sure! The results highlight "whether an automated approach that would still require micromanaging and adjusting several variables by the human researcher would, in fact, be more efficient an approach compared to the same tasks performed manually by human labour"
Out of Context! Managing the Limitations of Context Windows in #ChatGPT-4o Text Analyses https://doi.org/10.46298/jdmdh.15090 #DigitalHumanities #TextAnalysis #LLM #ArtificialIntelligence #GLAMR
EpisciencesOut of Context! Managing the Limitations of Context Windows in ChatGPT-4o Text AnalysesIn recent years, large language model (LLM) applications have surged in popularity, and academia has followed suit. Researchers frequently seek to automate text annotation - often a tedious task – and, to some extent, text analysis. Notably, popular LLMs such as ChatGPT have been studied as both research assistants and analysis tools, revealing several concerns regarding transparency and the nature of AI-generated content. This study assesses ChatGPT’s usability and reliability for text analysis – specifically keyword extraction and topic classification – within an “out-of-the-box” zero-shot or few-shot context, emphasizing how the size of the context window and varied text types influence the resulting analyses. Our findings indicate that text type and the order in which texts are presented both significantly affect ChatGPT’s analysis. At the same time, context-building tends to be less problematic when analyzing similar texts. However, lengthy texts and documents pose serious challenges: once the context window is exceeded, “hallucinated” results often emerge. While some of these issues stem from the core functioning of LLMs, some can be mitigated through transparent research planning.
)
on some research using Concept Mover's Distance (CMD or CMDist): a simple way to measure concept engagement in texts by incorporating word embedding models. 
New working paper: "Evaluating Embedding Models for Clustering Italian Political News"
Potential terrorists can be identified from social media posts, new research shows
An NLP-Based System for Detecting Depression Levels through User Comments on Twitter (X)
Attention Linguistics & Digital Humanities students! 
Don't miss out – registration is open until June 16th! 