Pavel Kuzko ← The long version
Case study 03 / Bug Free Immigration

Replacing manual applicant scoring with RAG

Co-founder and Product LeadBug Free ImmigrationLondonJun 2023 – Nov 2024

I co-founded an immigration consultancy and built its AI applicant evaluation product. The numbers below are real. The company closed anyway, which is its own lesson, and I am happy to talk it through.

8% → 18%
lead to client conversion after it shipped
+45%
lead qualification accuracy
40 → 8
hours of manual work a week

The problem

In an immigration consultancy, every lead has to be assessed before anyone knows whether the firm can help: what route fits, how strong the case is, whether it is worth a consultant’s hour. That assessment was manual. Consultants spent their week reading enquiries and CVs, and the good leads waited in the same queue as the hopeless ones.

The decision: ground the model in our own history

A general model knows immigration rules in the abstract. What it does not know is how cases like this one actually went. We had that: around 10,000 historical visa cases. So I built the scoring as retrieval over those cases, feeding the LLM assessment with the closest precedents instead of asking it to judge from first principles.

This is exactly where RAG earns its place: the answer has to be grounded in your data to be right.

Humans on the edge cases

The model scores every lead. Consultants review the scores and overrule the edge cases. That split matters: the model removes the reading, the people keep the judgement where a wrong call costs a client their application.

Automating the rest of the funnel

Once scoring worked, the bottleneck moved. I automated the rest of the funnel around it:

  • intake,
  • CV evaluation,
  • CRM workflows,
  • and nurture for leads that were not ready yet.

Result

Conversion went from 8% to 18%. Lead qualification accuracy went up 45%. Manual operations went from 40 hours a week to 8.

And the company still closed. Good unit economics on one part of the funnel do not save a business on their own; that is the lesson I took from it, and the one I bring to every product decision since.

Let the model do the reading. Keep people on the decisions where a mistake costs someone their case.

LLMRAG over 10,000 casesCRM automationhuman in the loop

Hiring for something like this?

I am a Senior AI Product Manager based in Belgrade, open to remote roles or EU relocation with two weeks notice.