DEV Community: LogicWiz Admin

How to Test Whether Candidates Can Spot Mistakes in AI-Generated Answers (an Interview Round You Can Run)

LogicWiz Admin — Thu, 08 Oct 2026 07:08:11 +0000

Imagine a candidate reading five lines an AI wrote about Python default arguments. The answer is fluent and mostly right. She nods and says, "Looks good to me." Line five is wrong. You never asked her to write code, and you've just learned something a coding test would have missed.

Short answer: give the candidate a confident, well-structured AI answer with exactly one planted technical error, ask them to review it out loud, and score whether they find the line, explain why it's wrong, and leave the correct lines alone. A shared document and a scorecard are enough. Below are the steps, a worked example, the scoring, and the tools that can help.

Why this skill matters now

A Sonar survey of more than 1,100 developers, published in January 2026, found that developers estimated AI wrote 42 percent of the code they committed. Of them, 96 percent didn't fully trust it to work correctly, only 48 percent said they always verify it before committing, and 38 percent said reviewing AI code takes more effort than reviewing a colleague's.

IEEE Spectrum reported in September 2026 on teams coping with the volume. Synthesia's CTO said pull requests were up 120 percent year over year as of August, and 95 percent contained AI-generated code. Bonterra's CTO said proposed changes tripled within three months of adopting AI and review times tripled too. Temporal's "Send Back" policy rejects agent-written code unless the engineer explains, in their own words, the agent's design choices and how the code handles unusual conditions.

A June 2026 arXiv paper by Vahid Garousi argues that oversight of AI output is mandatory: engineers must review, validate and sometimes rework what the AI produces. It's a synthesis of eight sources, seven of them practitioner writing, so it's a framing, not a measurement.

You'll also see claims that engineers now spend more hours reviewing than writing. The only figures I found came from a post that didn't name who ran the survey, so I've left them out. The sources above support something narrower: AI output needs a human check, and many developers admit they don't always do it.

For interviewers, "can you write this function?" is easy to answer with an assistant open. "Can you tell when this confident answer is wrong?" is harder to fake, and it's the job.

The round, step by step

It takes about ten minutes, as a screening call or a block inside a longer interview.

What you hand the candidate

Give them a question they could meet in the role, then an AI-style answer of five to seven numbered lines. Numbers let them point. Follow these rules:

Exactly one line is wrong. If you can't defend that, rewrite it.
The wrong line is a claim a busy person would accept, such as a convenient rule or a detail that's almost right.
Every other line is correct, and at least one correct line sounds fussy or surprising. That's your decoy.
All lines share the same tone and length, with no hedging.
You've checked every line yourself. An AI can draft the answer, but it sometimes plants a mistake you didn't ask for.

How long, and what to say

Allow one minute for the brief, three for a silent read, four for discussion and two for probes. Say this:

"An AI wrote this answer. It sounds confident, and one line contains a technical mistake. Review it the way you'd review a colleague's pull request. Think out loud, and tell me which line you'd send back and why."

Then stop talking. Don't say "good" or "close" while they work.

How to push back without giving the answer

React the same way whether the candidate is right or wrong. If they approve everything, or pick a line, use these probes in order:

"How sure are you? What would you check before approving this?"
"Pick the line you trust least. How would you test it?"
"Say this ships and breaks in production. Where do you look first?"

If they attack a correct line, ask, "What would you run to check that?" Strong candidates often talk themselves out of it. Never name the line.

What to write on the scorecard

Record the line they flagged first and the line they ended on, the reason in their own words, how many probes you used, whether they proposed a check (a test, a doc lookup, a quick run), and whether they criticised a correct line and how confidently.

A worked example

The question: a teammate's function keeps remembering items between calls, and the signature is def add(item, bucket=[]). Why?

Python evaluates default argument values once, when the def statement runs, and not on each call.

So bucket=[] creates a single list that every call without that argument shares.

Items added in one call therefore show up in later calls.

The usual fix is to default to None and create the list inside the function with if bucket is None: bucket = [].

This only affects lists. A dict or a set as a default is safe, because Python makes a fresh copy of those on each call.

Line 5 is the mistake. Python copies nothing. Dicts and sets are mutable too, so a {} or set() default is created once and shared exactly like the list. I ran all three in Python 3 and each carried data from the first call into the second.

A good reason: "Dicts and sets are mutable and created once at definition time, so they share state across calls just like the list." It names the mechanism. A weak reason: "dicts and sets are slower than lists, so they shouldn't be defaults." The candidate may have found the line by instinct, but the reason has nothing to do with the bug.

Lines 1 and 4 are the decoys, and both are true. Line 1 trips people who half-remember the rule and "correct" it to "evaluated on every call", which is backwards. A candidate who confidently attacks either has flagged a correct line, which is a different result from missing line 5. Write it down separately.

How to score it

Use a 0 to 3 scale and keep the notes beside it.

What happened	What it tells you	Suggested score
Caught the line and gave the right reason	Reads each claim and understands the mechanism	3
Caught the line, reason missing or wrong	Good instinct, may be pattern matching	2
Flagged a correct line, missed the real one	Confident criticism with no checking	0
Missed it, approved everything	Trusts fluent output	0

If a candidate needed a probe, drop them one level and note which probe. If they say "I can't tell for sure, here's the test I'd write", give that credit in the notes even when the line was wrong. A confident wrong critique still scores as a miss, however articulate.

Common mistakes interviewers make

Choosing trivia. If the error depends on an obscure flag, you're testing memory. Pick something the candidate would meet on the job.
Planting two errors by accident. Test every line, and have a colleague read it cold.
Leaking through reactions. A nod or "mm-hm" near the right line gives it away.
Rewarding the longest critique. Score the line and the reason, whatever the length.
Reusing one answer for every candidate. It will spread, so keep a few variants.

Tools that can run or support this

You don't need a tool for the version above. Here's what the main platforms say on their own pages, read on 7 October 2026.

HackerRank describes Chakra as an AI interviewer. A HackerRank post from 5 August 2026 says AI fluency is tested with scenarios such as reviewing AI-suggested code, catching a subtle bug a model introduced, and deciding whether a generated solution is good enough or needs a rewrite. Scoring runs across several dimensions, not a single pass or fail. TechCrunch reported on 5 October that Chakra gives candidates real repositories with an AI assistant, asks follow-up questions about their reasoning, and has run more than 500,000 interviews.

Codility's page on AI in technical assessment asks whether an engineer can judge if AI-generated code is correct, efficient and appropriate. It looks at whether a candidate engages with AI output critically or accepts the first answer, and it describes AI-generated follow-up questions that check whether candidates understand their own solution.

CoderPad's AI Fluency docs say interviewers can watch how a candidate prompts, iterates, evaluates AI output and applies judgment, and that reviewers should check whether the candidate verified the output. The AI Fluency page is aimed at non-technical candidates. The docs give little detail on scoring criteria.

CodeSignal's blog post describes an assistant called Cosmo and gives hiring teams a transcript of the candidate's AI interactions plus a session replay, which shows whether candidates ask informed questions, iterate well or lean on suggestions. It doesn't describe a planted-error review.

Placed.dev lists a GenAI coding round where the AI gives you code that looks right but isn't, and you find the bug, fix it and explain the change. MockIF describes a debugging round with a planted bug and a voice AI interviewer who scores your process. The MockIF page doesn't mention reviewing AI-written answers.

CloudApper's AI Recruiter page describes scenario-based questions and configurable scoring, and its article on AI in interviews is about detecting live AI help. None of the CloudApper pages I opened describes giving a candidate AI output to review. Kalpita's homepage lists Kalpita AI Interview as an enterprise AI product with no detail on how it assesses candidates, so I can't say how it handles this.

LogicWiz Interviews is one more option. Its inverted round has the AI write a confident, well-structured answer with a subtle technical error planted in it and ask the candidate to review it. If they miss the error, the AI pushes back gently without giving it away. A candidate who confidently criticises the wrong thing scores as a miss, and a structured scorecard records the result. Candidates can practise free with a free account: 3 AI sessions a day, no card, 16 tracks and 100 questions, by voice or text. There's also a free five-minute game, Spot Aria's Mistake, where you tap the wrong line across five rounds and then pick why. The question bank is small, and LogicWiz isn't a full coding-environment assessment tool. If candidates need to work in a repository, the platforms above suit that better.

Common questions

What is an inverted interview round?

The AI or interviewer supplies the answer and the candidate does the checking. The answer sounds confident and contains one planted error.

How many errors should the answer contain?

One. A single error keeps the scoring clean and lets you tell a miss from a wrong critique. Add more lines, not more mistakes.

Should I tell candidates there's a mistake?

For a ten-minute round, yes. Without the hint you mostly measure suspicion. For senior roles you can drop it and score whether they ask what to verify.

Can I use ChatGPT or another AI to write the answer?

For a draft, yes, but check every line yourself. An AI asked for a subtle mistake sometimes plants two, or one that's actually correct.

Is this round enough to make a hiring decision?

No. It shows how someone reviews and verifies. Pair it with a build or debug task.

Sources I read

All opened and read on 7 October 2026.

Sonar press release on the AI coding verification gap, 8 January 2026: https://www.sonarsource.com/company/press-releases/sonar-data-reveals-critical-verification-gap-in-ai-coding/
IEEE Spectrum, Aaron Mok, "AI Slop Is Changing How Engineers Review Code," 8 September 2026: https://spectrum.ieee.org/ai-code-review-software-engineers
Vahid Garousi, "Human Oversight and Overload," arXiv 2606.05770, 4 June 2026: https://arxiv.org/abs/2606.05770
HackerRank, "How Chakra by HackerRank works," 5 August 2026: https://www.hackerrank.com/blog/how-chakra-by-hackerrank-works/
TechCrunch, Jagmeet Singh, 5 October 2026: https://techcrunch.com/2026/10/05/hackerranks-ai-interviewer-offers-a-glimpse-into-what-job-interviews-could-become/
HackerRank Chakra page: https://www.hackerrank.com/chakra
Codility: https://www.codility.com/ai-technical-assessment/
CoderPad: https://coderpad.io/resources/docs/ai-fluency/ and https://coderpad.io/ai-fluency/
CodeSignal: https://codesignal.com/blog/introducing-ai-assisted-coding-assessments-interviews/
Placed.dev: https://placed.dev/
MockIF: https://www.mockif.com/debugging-interview-practice
CloudApper: https://www.cloudapper.ai/ai-recruiter/ and https://www.cloudapper.ai/talent-acquisition/detect-ai-tools-in-interviews-verify-candidate-authenticity/
Kalpita Technologies: https://www.kalpitatechnologies.com/

Free Coding Quiz Games for College Students in India (2026), Compared Honestly

LogicWiz Admin — Thu, 08 Oct 2026 07:07:30 +0000

Search for free coding quiz games and the results mix three kinds of thing: multiple choice quizzes that take a few minutes, timed contests where you write code against thousands of people, and company hackathons that run for a day. Land on the wrong kind and it's easy to quit after one try.

I run LogicWiz, which has its own free coding game, so I'm biased. I've put it in the list and said where it's weaker. Everything below comes from each product's own page, opened on 7 October 2026. If a page didn't say something, I left it out or said so. Listings change weekly, so check the page before you plan around a date.

What to look for

What "free" means. Free to enter, free forever, a free preview with a paid full set, and a free site with a fee on some listings are different things.

Format. A five minute quiz, a two hour contest and a 24 hour hackathon take very different amounts of your week.

Reading or writing code. Some tests ask you to recall facts, some to write a solution, and some to judge code someone else wrote.

Sign-in friction. Google login, a full registration, or nothing at all.

What you get at the end. A rank among thousands, a certificate, or instant answers with explanations.

Unstop quizzes: the biggest list, mostly not coding

Unstop's quiz page says it has "5606+ Online Quizzes & MCQ Tests" and lets you filter by team size, payment and category. The listings I saw were college and company events, online or in person, for individuals or teams. Some showed an entry fee (49 and 300 in the ones I saw) and some showed none. Many were business or economics quizzes.

Who it suits: a student who wants college fest quizzes with prizes and will filter for the tech ones. The limit is the digging: coding quizzes are a slice of a big mixed list, so check each listing's fee and eligibility.

Internshala quizzes and competitions: free listings, thin on coding quizzes

Internshala's quiz page says skill based quizzes can cover logical reasoning and coding. It listed two quizzes when I read it: TCS InQuizitive 2026 and the Krishi Medha Quiz 2026. Both were online and free, running to 31 December 2026. Neither listing says it is a coding quiz. The wider competitions page tags each hackathon and contest as Free or Paid.

Who it suits: students who want hackathons and internships on one site. The limit is that the quiz page had little to play when I looked.

CodeChef: weekly contests where you write code

CodeChef's contests page describes rated coding challenges every week. It showed Starters 259 running, with a Monday Munch DSA Challenge and Placement Prep Weekends coming up. A recent Starters round had 35,518 participants. A Skill Tests section lists Python, Java, C, C++, SQL and JavaScript tests, lasting 45 minutes to 1 hour 30 minutes.

Who it suits: a student comfortable writing code under a clock who wants a rating. The page doesn't say what the skill tests cost, and it shows an "Upgrade to Pro" button, so check before you assume they're free. A beginner may find a first round discouraging.

LeetCode contests: one weekly round, one every two weeks

LeetCode's contest page says there's a contest every week and that you can see your ranking. It listed Weekly Contest 523 on Sunday 11 October at 8:00 am IST and Biweekly Contest 193 on Saturday 10 October at 8:00 pm IST. Past contests showed scores out of four, so a round has four problems, and past rounds can be replayed as virtual contests. You sign in to see your own results. The page doesn't mention price.

Who it suits: a student already practising data structures and algorithms for placements. The limit is that every round is code writing, with no lighter format.

Naukri Code360: free weekly contests with a beginner series

The page title says "Free Coding Competitions." It lists rated weekly contests (Weekly Contest 251 on 8 October at 8:31 pm IST, four problems) and a separate Beginner Contest series, with Beginner Contest 123 a week out. Top ranks earn badges, and you register for each contest.

Who it suits: a student who wants a contest with a beginner track. The limit is that the page doesn't say how long a round lasts, so check the details before you register.

Codeforces: the serious competitive programming option

Codeforces runs rounds split into divisions, plus Educational rounds. Round 1125 (Div. 3) started at 8:15 pm IST on 7 October, ran two and a half hours, and had over 30,000 people registered. The page marked it as open to unrated entries. Past rounds can be played virtually, and you register to take part.

Who it suits: a student who wants hard problems and a global rating. The limit is difficulty. Read the division before you register, because a Div. 1 round can be a long evening for someone who has been coding for a month.

Underrated Coder: free MCQ quizzes with a certificate quiz

Underrated Coder's quiz page promises free coding quizzes in Python, Java, C++, HTML, CSS, JavaScript, React and SQL, with instant results and detailed explanations. It lists 12 or more quizzes, each marked by level. The Python quiz page describes a certification quiz of 15 multiple choice questions in 5 minutes. Starting it asked me to log in with Google, GitHub or email, so I stopped there.

Who it suits: a first or second year student who wants a quick language check. The limits: it's multiple choice, so it tests recall, and it needs a login. The site's menu also lists live classes and 1:1 mentorship.

LearnFullStack: free games and quizzes, heavy on web development

LearnFullStack calls itself 100 percent free. Its pages list 135 games and 249 quizzes, mostly HTML, CSS, JavaScript and React, with 10 Python games and one Python quiz. I opened the Python Basics game without signing in. It was a 25 challenge set with a code box and a Run Code button. The homepage mentions certificates for finished roadmaps, and the menu has a leaderboard.

Who it suits: a beginner learning web development who wants to type code in small steps. The limits: it's light on SQL (MySQL has 2 games and 1 quiz), and I saw no cloud or data engineering games. I didn't log in, so I can't say what saves your progress.

HackerRank and HackerEarth: company hackathons

Both contest pages were mostly company events. HackerRank listed Orchestrate October 2026, a 24 hour hackathon to design, build and ship an agent, starting 24 October at 6 pm IST. Internshala lists the same event as online and free. HackerEarth's Compete page showed live and open hackathons from Booking Holdings, Airwallex and Godrej Capital. Neither contest page showed a price.

Who it suits: a student with a free weekend who wants to build a project. If you want a ten minute test, start elsewhere.

Spot Aria's Mistake (LogicWiz): free, five minutes, judge instead of write

Spot Aria's Mistake is the game I built, so weigh this section accordingly. It takes about five minutes on a phone and has five rounds. An AI called Aria answers a tech question line by line with total confidence, and exactly one line is wrong. You tap the wrong line, then pick why. You get points for the line, a bonus for the reason and a speed bonus.

There are eight tracks: AI, Python, Frontend, Backend and APIs, SQL and databases, Data engineering, Cloud, DevOps and Git, and DSA and OOP, plus an easy Kids Playground track. A live scoreboard sits alongside a College league that ranks colleges, and everyone who finishes gets a certificate. It's open until 19 October 2026, and the top 10 players win free LogicWiz goodies delivered home.

Who it suits: a student who wants a short, phone friendly game that tests whether you can catch a plausible mistake. That matters because more of the code you meet now starts as an AI draft.

The limits: the question bank is small, and there's no code editor, so it tests reading and judging answers, and it never asks you to write code. You need a Google sign-in. Each person gets one attempt, though a player who scores 0 gets up to 3 rematches with new questions.

Side by side

Option	Free?	Format	Sign in	Best for
Unstop quizzes	Some listings free, some show a fee	College and company quizzes, solo or team	Page didn't say	Fest quizzes
Internshala	Listings tagged Free or Paid	Quizzes, contests, hackathons	Register or login	Competitions and internships
CodeChef	Not stated on page	Weekly rated contests, skill tests	Login or sign up	Writing code under a clock
LeetCode	Not stated on page	Weekly and biweekly, 4 problems	Sign in for your results	DSA and placement prep
Naukri Code360	Titled free	Weekly rated, beginner series	Register per contest	Beginner contest track
Codeforces	Page didn't say	Rounds of about 2 to 3 hours	Register	Hard problems, global rating
Underrated Coder	Described as free	MCQ quizzes, 15 questions in 5 minutes	Login required	Quick language check
LearnFullStack	Described as 100% free	Games with a code box, quizzes	A game opened without login	Web development beginners
HackerRank, HackerEarth	Varies by event	24 hour company hackathons	Login	Building a project
Spot Aria's Mistake	Free	5 rounds, about 5 minutes, spot the wrong line	Google	Reading and judging answers

I dropped four others. GeeksforGeeks' weekly contest page says it's temporarily paused. Kaggle's page sat behind a browser check, so I didn't read it. Coding Adda's ExamZeta gives the first five questions free and needs a subscription for full sets. HackIndia's games address redirected to a skills page.

How to pick

Five minutes free: a quiz or short game. Codeforces rounds alone run two hours or more.
Want to practise writing code: CodeChef, LeetCode or Code360, starting with a beginner round.
Want to learn web basics by typing: LearnFullStack.
Want prizes from college events: filter Unstop and Internshala for tech.
Want to test whether you catch a confident wrong answer: Spot Aria's Mistake.

Common questions

What are the best free coding quiz games for college students in India?

It depends on the format. Underrated Coder and LearnFullStack describe their quizzes as free, CodeChef, LeetCode, Naukri Code360 and Codeforces run timed code contests, and Spot Aria's Mistake is a five minute game about spotting a wrong line.

Which of these needs no sign-in?

I could open a LearnFullStack game without signing in. Underrated Coder asked for a login before its quiz, the contest sites ask you to register, and Spot Aria's Mistake needs a Google sign-in.

Can I play on my phone?

Spot Aria's Mistake takes about five minutes on a phone. The other pages I read didn't say anything about phones, so I can't confirm it for them.

Is there a beginner friendly contest?

Naukri Code360's page lists a separate Beginner Contest series. CodeChef and LeetCode didn't label a beginner track on the pages I read.

Do any of them give a certificate?

Underrated Coder's Python quiz is a certification quiz, LearnFullStack lists certificates for finished roadmaps, and every Spot Aria's Mistake player who finishes gets one.

Sources I read

All pages opened on 7 October 2026.

Can you spot the AI's mistake? Try this 5-line JSON answer

LogicWiz Admin — Mon, 05 Oct 2026 03:34:02 +0000

Here is a short answer an AI wrote about JSON. Four lines are true and one is not. Decide which before you scroll.

What are the basic rules of JSON?

JSON is a plain-text format for structured data, and most languages can read and write it.

Object keys must be strings in double quotes, so {name: 1} is not valid JSON.

Values can be strings, numbers, true, false, null, arrays or objects.

You can add a comment on its own line with //, and parsers skip it like whitespace.

A trailing comma after the last item, as in [1, 2,], is not allowed.

It is line 4. Standard JSON has no comments. RFC 8259 defines none, and JSON.parse throws on that line. Some tools accept comments as an extension, which is why the claim sounds fine.

The trap is line 5. It sounds like a fussy made-up rule, and it is true. Calling a correct line wrong is just as costly as missing the wrong one.

Why this is worth practising

A lot of the code and explanations we read now start as an AI draft. Producing an answer is only half the job. The other half is reading a fluent answer and catching the one sentence that would break production. Normal quizzes do not train that, because the question itself tells you something is being tested. Here the wrong line looks exactly like the right ones.

Three habits that help:

Read for the claim, not the tone. Every line sounds equally sure.
Distrust the convenient rule. Line 4 describes what we wish JSON allowed.
Say the reason to yourself first. If you cannot say why a line is wrong, you are guessing.

Want more rounds?

We turned this into a free five-minute game, Spot Aria's Mistake: five rounds, one wrong line in each, 20 seconds to tap it, then pick why it is wrong. Eight tracks, from Python and SQL to AI and DSA. There is a practice round that needs no sign-in: arena.logicwiz.ai.

Disclosure: I work on LogicWiz, which makes the game.

Originally published on LogicWiz.

MCP Explained With an Example

LogicWiz Admin — Thu, 01 Oct 2026 09:19:53 +0000

The AI does not know your tools exist. That sentence is the reason MCP exists, and most
explanations skip it.

A language model can only produce text. It cannot read your database, send an email, or
check a calendar. If you want it to do those things, something has to tell it what tools
are available, let it ask for one, run the tool for it, and hand the result back. Before
MCP, every app wrote that plumbing its own way, so a tool built for one assistant did not
work with another.

MCP, the Model Context Protocol, is a shared way of doing that plumbing. Any tool that
speaks MCP works with any assistant that speaks MCP. One plug for every tool.

The three parts

Host: the app the person is using. A chat assistant, a code editor, your own agent.
Client: the small piece inside the host that talks MCP. Usually you never write it.
Server: the thing that owns the tools. A server for your database, a server for GitHub, a server for the file system. Each one publishes a list of what it can do.

The model itself is not any of these. It sits behind the host, sees the list of tools,
and decides which one to ask for.

The handshake, with the real messages

Here is what happens when a host connects to a tiny weather server. The messages are
JSON-RPC, which just means a JSON object with a method name and parameters.

1. Connect and initialize. The client says hello and both sides agree on what they
support.

{"jsonrpc": "2.0", "id": 1, "method": "initialize",
 "params": {"protocolVersion": "2025-06-18",
            "clientInfo": {"name": "my-agent", "version": "1.0"}}}

2. Ask what tools exist. This is the step that matters. The model has no idea what
the server can do until this list comes back.

{"jsonrpc": "2.0", "id": 2, "method": "tools/list"}

The server replies with every tool, its description, and the input it needs:

{"jsonrpc": "2.0", "id": 2, "result": {"tools": [
  {"name": "get_forecast",
   "description": "Get the 3-day weather forecast for a city. Use this when the user asks about upcoming weather, not current conditions.",
   "inputSchema": {"type": "object",
                   "properties": {"city": {"type": "string"}},
                   "required": ["city"]}}
]}}

3. The model chooses. The host puts that list in front of the model along with the
user's question, "Will it rain in Pune this weekend?" The model reads the descriptions
and replies that it wants to call get_forecast with {"city": "Pune"}.

4. Call the tool. The client sends the request. The model does not run anything; the
server does.

{"jsonrpc": "2.0", "id": 3, "method": "tools/call",
 "params": {"name": "get_forecast", "arguments": {"city": "Pune"}}}

5. Get the result back.

{"jsonrpc": "2.0", "id": 3, "result": {"content": [
  {"type": "text", "text": "Sat: rain, 24C. Sun: cloudy, 26C. Mon: clear, 28C."}
]}}

6. The model answers the person. The host feeds the result back to the model, and
the model turns it into a sentence: "Yes, Saturday looks wet. Sunday should be dry."

Ask, choose, run, reply. That is the entire protocol in practice.

The mistake that breaks most MCP setups

Look at step 2 again. The description field is the only thing the model has when it
decides which tool to call. It is not documentation for humans. It is the interface.

A server with two tools described as "Get weather" and "Get weather data" will make the
model guess, and it will guess wrong about half the time. Then you spend an afternoon
blaming the model when the real problem was one vague sentence.

Write descriptions the way you would brief a new colleague: what the tool does, when to
use it, and when not to. The forecast tool above says "upcoming weather, not current
conditions" for exactly that reason. If you also have a get_current_weather tool, that
one line is what stops the model from mixing them up.

When you do not need MCP

If your agent has three tools and you wrote all three yourself, plain function calling is
simpler. MCP earns its place when tools and assistants are built by different people:
you want to plug a database server someone else wrote into an assistant you did not
build. That is the N by M problem, N assistants times M tools, and MCP turns it into
N plus M.

Where to go from here

The LogicWiz GenAI course animates this handshake so you can watch each message go across:
Enter MCP, one plug for every tool,
the MCP handshake, and
when, and when not, to use MCP. There is also a chapter on
A2A, the protocol for agents talking to other agents. The whole course is completely free, with no card. Chapters one to three open without an account, and from chapter four a free account keeps you going.

Originally published on LogicWiz.

RAG Tutorial for Beginners: Build a Retrieval Pipeline in Python

LogicWiz Admin — Wed, 30 Sep 2026 14:25:08 +0000

RAG stands for retrieval augmented generation. Strip the jargon and it is one idea: the
model does not know your documents, so before you ask it a question, you look up the
relevant passage yourself and paste it into the prompt. Retrieve, augment, generate.

That is the whole trick. Everything else in a RAG pipeline (embeddings, chunking, vector
databases, rerankers) exists to make the "look up the relevant passage" step work on
thousands of pages instead of one.

This tutorial builds the pipeline from scratch in Python. No LangChain, no vector
database, one file. By the end you will know what those tools are doing when you do
reach for them.

Why the model needs help in the first place

A language model answers from what it saw during training. Ask it about your company's
refund policy, last week's meeting notes, or a PDF you were sent this morning, and it
has two options: admit it does not know, or produce something confident and wrong. Most
models pick the second.

You could paste the whole document into every prompt. That works until the document is
longer than the context window, or you have five hundred documents, or you are paying
per token and the document is 40 pages. RAG is the fix: send only the few paragraphs
that matter for this question.

The pipeline in one picture

your documents
   │
   ▼
1. chunk      split each document into passages of a few hundred words
   │
   ▼
2. embed      turn each passage into a list of numbers that captures its meaning
   │
   ▼
3. index      store the passages and their numbers so you can search them
   │
   ▼            user question
4. retrieve   embed the question, find the passages whose numbers are closest
   │
   ▼
5. augment    put those passages into the prompt, above the question
   │
   ▼
6. generate   the model answers using the passages it was just shown

Steps 1 to 3 happen once, when you load the documents. Steps 4 to 6 happen on every
question.

Step 1: chunk the documents

A passage has to be small enough that a handful of them fit in the prompt, and large
enough to still make sense on its own. A few hundred words is the usual compromise.

def chunk(text: str, size: int = 300, overlap: int = 50) -> list[str]:
    words = text.split()
    chunks = []
    start = 0
    while start < len(words):
        chunks.append(" ".join(words[start:start + size]))
        start += size - overlap
    return chunks

The overlap matters. Without it, a sentence that straddles a boundary is cut in half and
neither chunk contains the full thought.

Step 2: embed each chunk

An embedding is a list of a few hundred or a few thousand numbers. Passages that mean
similar things get lists that point in similar directions. That is what lets you search
by meaning rather than by exact words.

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from the environment

def embed(texts: list[str]) -> list[list[float]]:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=texts,
    )
    return [item.embedding for item in response.data]

Step 3: index

For a tutorial, the index is a Python list. A vector database is this list plus fast
search over millions of rows. You do not need one until the list gets slow.

documents = {
    "refunds.md": open("refunds.md").read(),
    "shipping.md": open("shipping.md").read(),
}

index = []  # each entry: (source, chunk_text, embedding)
for source, text in documents.items():
    chunks = chunk(text)
    for chunk_text, vector in zip(chunks, embed(chunks)):
        index.append((source, chunk_text, vector))

Step 4: retrieve

Embed the question with the same model, then find the chunks whose vectors point the
same way. Cosine similarity is the standard measure.

import math

def cosine(a: list[float], b: list[float]) -> float:
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(y * y for y in b))
    return dot / (norm_a * norm_b)

def retrieve(question: str, k: int = 3) -> list[tuple[str, str]]:
    q_vector = embed([question])[0]
    scored = [
        (cosine(q_vector, vector), source, chunk_text)
        for source, chunk_text, vector in index
    ]
    scored.sort(reverse=True)
    return [(source, chunk_text) for _, source, chunk_text in scored[:k]]

Steps 5 and 6: augment and generate

Put the retrieved passages in the prompt, tell the model to answer from them, and ask.

def answer(question: str) -> str:
    passages = retrieve(question)
    context = "\n\n".join(f"[{source}]\n{text}" for source, text in passages)
    prompt = (
        "Answer the question using only the passages below. "
        "If the passages do not contain the answer, say so.\n\n"
        f"{context}\n\nQuestion: {question}"
    )
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content

print(answer("How many days do customers have to request a refund?"))

That is a working RAG pipeline. Run it against two small markdown files and you will see
the model quote your refund policy instead of inventing one.

The three places it goes wrong

Every RAG bug in production is one of these.

The right passage was never retrieved. The question used different words from the
document ("money back" versus "refund"), or the chunk boundary split the answer, or the
answer needed two passages and you only fetched one. Print the retrieved passages before
you blame the model. Nine times out of ten the model never saw the answer.

The right passage was retrieved and the model ignored it. Usually the prompt is too
soft. "Use the passages below" is a suggestion; "answer only from the passages, and say
'not in the documents' otherwise" is an instruction. Test with a question the documents
cannot answer and check that the model says so.

The passages are stale. Documents changed and the index did not. The index is a
cache, and it needs the same care as any cache: rebuild it when the source changes.

Where to go from here

Once the basic loop works, the upgrades are all in the retrieve step: hybrid search
(keyword match plus embeddings), reranking the top 20 down to the best 3 with a second
model, and metadata filters so a question about shipping never retrieves a refund
passage. Each one fixes a specific failure you will hit, so add them when you hit it,
not before.

The LogicWiz GenAI course covers this chapter with an animation you can step through
and a lab that runs in the browser: Why Nova needs RAG,
embeddings and semantic search,
chunking, and indexing and retrieval.
The whole course is completely free, with no card. Chapters one to three open without an account, and from chapter four a free account keeps you going.

Originally published on LogicWiz.