Organisations spend around 60% of their research time processing open-ended responses. Not analysing them - processing them. That gap is where the decision gets delayed.
Why this problem found me
We did not go looking for it. While working on the products we hit the same wall directly: piles of open-ended survey text, no efficient way to turn them into something a decision could rest on. Most AI tooling in this space predicts a number - regression, classification - and skips perceptual understanding entirely, which is where the actual insight lives.
The third framing matters most. Small institutes cannot afford a data analyst, and non-technical users cannot interpret raw survey data. The tooling exists; the access does not.
Two approaches that did not work
The first attempt was traditional NLP - sentiment analysis, structural topic modelling, TF-IDF keyword extraction. It lacked contextual understanding, needed constant manual tuning, and did not scale.
The second combined machine learning models with NLP for classification and topic modelling. Accuracy improved, but the system stayed rigid: no adaptability, no real-time insight. Better numbers, same ceiling.
The dual-stream architecture
The third iteration worked: a GPT-based LLM fine-tuned for accuracy, with retrieval-augmented generation layered on for real-time context. Three components - automated question generation and interpretation, AI-driven analysis with a self-learning feedback loop, and interactive visualisation through D3.js and Plotly with a chat interface on top.
Trained against a corpus of 100,000+ annotated responses, it reached 95% overall accuracy, cut manual processing by 60%, reduced processing time by a further 40% through the feedback loop, and held roughly 85% F1 on sentiment and summarisation.
The part we would tell a client
Write down the approaches that failed. A result with no failed iterations behind it is either luck or a very small problem. The progression from rule-based to deep learning to fine-tuned LLM plus RAG is the actual finding - it tells the next person where the ceiling of each method is, which is worth more than the final accuracy figure.