Real Interview Questions for the Agentic AI Engineer at Tata Consultancy Services (TCS)
NewInterview Q&A
Q&A-1What is the step-by-step architecture of an enterprise RAG system, and how does data flow from ingestion to vector storage and response generation?
Reported in Tata Consultancy Services AI Engineer interview questions ...
The 90-second answer
I'd split this into two pipelines that share storage. An offline ingestion pipeline turns source documents into chunks, embeddings and metadata. An online query pipeline authenticates the user, retrieves, filters, assembles context and generates a cited answer. The vector index is one component of that. In an enterprise build, the parts that decide whether it survives audit are the ACL filter applied inside the retrieval query and the version provenance on every chunk.
Before designing, I'd confirm the sources (SharePoint, Confluence, SAP documents, ticketing), the change rate, data residency rules, and the latency target. Assumed figures for the rest of this answer, all hypothetical: 2 million documents, about 12 chunks per document, 5,000 documents changed per day, p95 end to end under 4 s.
Ingestion path
- Connectors read change feeds or delta queries from each source. Full re-crawls are for backfill only.
- Parse and normalise. Layout-aware PDF extraction, tables kept as markdown, OCR for scans. Strip headers, footers and boilerplate.
- Chunk on heading structure, roughly 300 to 500 tokens with about 10% overlap. Each chunk carries doc_id, version, section path, ACL groups, classification and effective date.
- Embed in batches with a pinned model version, and store the model_id on every vector. Mixing embedding models in one index silently destroys similarity scores.
- Upsert chunks keyed by content hash, and delete the chunks of superseded versions in the same job, so stale text cannot be retrieved.
Storage
A vector index (HNSW is the usual default) holds embeddings plus filterable metadata. Original chunk text and the document registry live in a relational or object store. A BM25 index runs alongside the vectors for hybrid search, because exact identifiers such as invoice numbers and part codes are poorly served by embeddings.
Query path
- Authenticate and resolve the user's groups from the identity provider at request time. Do not use group membership cached in the index.
- Embed the query with the same model_id as the corpus.
- Retrieve top 50 from vectors and top 50 from BM25, with the ACL filter pushed into both queries. Fuse with reciprocal rank fusion, then rerank the fused list down to 8.
- Build the prompt with numbered sources and a fixed context budget, about 6k tokens here.
- Generate at low temperature and require inline citations like [1]. Post-check that every citation maps to a retrieved chunk. If not, retry once, then refuse.
- Log the trace: query, chunk ids, versions, model ids, and latency per stage.
Arithmetic check
2M documents × 12 chunks = 24M vectors. At 1024 float32 dimensions, that is 4 KB per vector, so about 96 GB of raw vectors before index overhead. Ingestion at 5,000 documents a day produces about 60,000 new chunks daily, so the embedding job is incremental and small. A full reindex is the expensive event, which is why the model_id check matters.
| Failure | How it shows up | Fix |
|---|---|---|
| ACL filter applied after top-k | Permission test users see documents they should not | Filter inside the index query |
| Superseded version served | Answer cites an old version_id | Delete old chunks in the upsert job, track version freshness |
| Mixed embedding models | Recall@50 drops after a model rollout | Store model_id, reindex fully before switching |
| Context truncation | Citation check fails on answers | Enforce the token budget before generation |
When not to use RAG
RAG handles knowledge that changes and must be cited. For stable style or behaviour, fine-tuning is the better tool. For questions that need an exact aggregate across records, query the SQL system directly and let the model narrate the result.
Curated: · Written: · Reviewed:
Q&A-2How will you handle restricted vs unrestricted questions before retrieval?
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
I keep two decisions apart. The first is whether a query touches restricted data, which is a routing hint. The second is what this caller may read, which is the security boundary. The boundary never depends on the model's judgement.
Identity comes from the authenticated token before the agent sees the question. I map OIDC groups to data classes (public, internal, customer_commercial, hr_restricted) and pass the agent a resolved principal, not a claim the user typed. Retrieval then runs with a metadata filter built from that principal. Restricted chunks never enter the candidate set for an unentitled caller, so there is nothing to redact afterwards.
The classifier comes second. A small, deterministic pass tags the query as unrestricted (general product or policy), restricted (named customers, pricing, HR, contracts) or ambiguous. Restricted and ambiguous queries go to the filtered path. Unrestricted queries may use the broad index, which holds only public and internal data. A misclassification costs recall or latency, not exposure, because the filter applies either way.
Pre-filter, not post-filter. Post-filtering after top-k is simpler and wrong. It leaks existence through result counts and timing, and it returns fewer than k chunks for restricted users, so answer quality varies by role.
Trace (hypothetical figures)
| Step | Value |
|---|---|
| Query | "What discount did Acme get in Q3?" |
| Principal | groups sales-emea |
| Classifier | restricted, class customer_commercial, confidence 0.94 |
| Filter | data_class IN ('public','internal') OR (data_class='customer_commercial' AND 'sales-emea' IN acl_groups) |
| Result | 6 chunks, all from the EMEA partition, none from hr_restricted |
| Same query, caller without the group | filter yields 0 chunks, agent replies it cannot access that, no hint whether Acme exists |
Trade-offs. ACL pre-filtering keeps recall honest but needs the ACL metadata indexed and synced from the source system. Routing adds a classifier hop, so for a small single-tenant corpus I skip routing and enforce the filter alone.
Failure modes and detection
- Classifier labels a restricted question as unrestricted, and the broad index answers it. The broad index contains no restricted classes, so this cannot leak content. I detect it by logging the classifier label against retrieved data classes and alerting on any restricted class in a broad-path response.
- Group membership changes in the IdP but the index is stale. Access is then wrong in both directions. I set a maximum sync lag (15 minutes is a reasonable starting target for this design) and run a nightly sample comparing index ACLs with the IdP.
- Filter syntax silently ignored by the vector store, so the filter matches everything. I guard this with a canary document in a restricted class that must never appear for a test principal, checked in CI and on a schedule in production.
The classifier is a cost and quality optimisation. The filter at the data layer is the control, and I would defend the design on that basis.
Curated: · Written: · Reviewed:
Q&A-3How can we handle large chunks in a RAG pipeline?
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
First I'd pin down which failure "large" causes, because the fixes differ. A chunk can exceed the embedding model's input limit, which is a hard failure: the request errors or the tail gets truncated without warning. It can also be too big to fit the generation budget once I pass k chunks to the LLM. And it can be big enough that its vector is an average over several topics, so it matches queries poorly. The first breaks ingestion. The other two show up as wrong or ungrounded answers.
My default: split on document structure first (headings, clauses, sections), then cap each child chunk at roughly 400 to 600 tokens, counted with the embedding model's own tokenizer, with 10 to 15 percent overlap. Index the children for retrieval. Store each child's parent section id, and when a child matches, send the parent to the LLM, capped at a fixed token size. Precise matching, enough context for generation.
Arithmetic. A 60-page policy PDF is about 30,000 tokens (hypothetical, but typical for dense English). Fixed 4,000-token chunks give about 8 vectors. A query whose answer sits in a 300-token clause gets compared against an embedding that is mostly the other 3,700 tokens. With 500-token children at 15 percent overlap you get roughly 70 vectors. Retrieve the top 5 children, expand them to their parents (about 2,000 tokens each, deduplicated, so maybe 3 parents), and the LLM sees around 6,000 tokens of targeted context instead of 20,000 or more of mixed text.
from tiktoken import get_encoding
enc = get_encoding("cl100k_base") # match the embedding model's tokenizer
def token_windows(text, max_tokens=512, overlap=64):
ids = enc.encode(text)
step = max_tokens - overlap
for start in range(0, len(ids), step):
yield enc.decode(ids[start:start + max_tokens])
if start + max_tokens >= len(ids):
break
That snippet cuts on token boundaries, which can split a sentence or a table row. In production I'd split on sentences or structural boundaries first, and only fall back to token windows for blocks that are still too large.
Trade-offs. Parent expansion costs context budget, so cap it and drop lowest-scoring parents first. Larger overlap improves recall at a cost in index size and duplicate hits. Late chunking (embed the whole document with a long-context embedding model, then pool token vectors per chunk) keeps cross-chunk context in each vector, but it needs a long-context embedder and a full re-embed on any document change. I'd reach for it only when chunks keep losing their referent, such as pronouns that point back into the previous section.
Failure modes and how I'd detect them
- Tables and code blocks split mid-structure. Detect by sampling chunks at ingest and flagging ones that open or close inside a fence or table.
- Overlap returns near-duplicate hits and crowds out distinct evidence. Detect by counting unique parent ids in the top-k.
- Chunks lose their heading context. Prepend the document title and heading path to each chunk before embedding.
- Parent expansion blows the prompt budget. Track context tokens per request; alert on p95 above the model's safe limit.
- Re-chunking leaves a stale index. Version the chunker config and store it with each vector, so a changed size or overlap forces a rebuild.
To confirm any of this worked, I'd run recall@5 on a labelled query set before and after the change, and a groundedness eval on final answers. Latency alone won't tell me whether retrieval got better.
Curated: · Written: · Reviewed:
Q&A-4What are Top-K and Top-P? Explain the difference and use cases.
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
Both are truncation rules applied to the next-token distribution before sampling. Top-K keeps the K most likely tokens. Top-P keeps the smallest set of most likely tokens whose cumulative probability reaches P. Both renormalize what's left and sample from it. Neither changes the model. They change which tail tokens can be picked. Top-K uses a fixed count, so it treats a sharp distribution and a flat one the same. Top-P uses a fixed probability mass, so the candidate set grows when the model is unsure and shrinks when it is confident.
For agents, I keep the planner and tool-selection steps conservative (low temperature, low top-P) because a sampled token inside a JSON tool call is a parser failure, not a stylistic choice. I only loosen sampling for the final user-facing prose.
Worked trace (hypothetical probabilities for the next token)
| Token | p | Cumulative |
|---|---|---|
| the | 0.50 | 0.50 |
| a | 0.20 | 0.70 |
| this | 0.10 | 0.80 |
| one | 0.08 | 0.88 |
| an | 0.05 | 0.93 |
| our | 0.04 | 0.97 |
| other | 0.03 | 1.00 |
Top-K=3 keeps {the, a, this}. Their mass is 0.80, so they renormalize to 0.625, 0.25 and 0.125. Top-P=0.9 keeps five tokens, because the cumulative is still 0.88 after one and only reaches 0.93 at an. It renormalizes over 0.93. On a flat distribution where the top ten tokens each hold 0.09, Top-P=0.9 keeps ten, while Top-K=3 still keeps three. That is the adaptive difference in one example.
Failure modes and how I'd detect them
- Top-K too small on open-ended generation gives repetitive, generic text. Too large lets low-probability tokens through and makes answers drift. Detect with a repetition rate and a human-rated sample on a fixed prompt set.
- Top-P near 1.0 disables the filter. Very low temperature combined with very low Top-P narrows the choice twice and can produce loops. Detect with output length distributions and loop detection in the harness.
- Tool-call corruption: sampling too loose breaks schema validity. Track schema-validation failure rate per model and setting, and retry or reject before execution.
- Provider differences: support and defaults vary by vendor and model, and some APIs do not expose Top-K at all. Check the current API reference for the exact parameter names, defaults and the order in which temperature, Top-K and Top-P are applied. Vendors commonly advise changing temperature or Top-P, not both, so I tune one and hold the other.
- Determinism: even at temperature 0, hosted models can return different outputs across runs. Do not rely on sampling settings for replay. Cache tool results and log the full prompt and parameters for audit.
When not to use them
If you need exact, repeatable output (extraction, routing, structured output), use greedy or near-greedy decoding with schema validation, not a larger Top-P. Sampling controls shape style and diversity. They are not a correctness mechanism.
Curated: · Written: · Reviewed:
Q&A-5What is temperature in LLMs?
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
Temperature is a sampling parameter. It divides the model's logits before the softmax: p_i = exp(z_i / T) / Σ_j exp(z_j / T). At T=1 the model's distribution is used unchanged. Lower T sharpens it toward the top token. Higher T flattens it, so lower-ranked tokens get sampled more often. Temperature does not change the weights, the knowledge, or the ranking of tokens. It changes how much of that ranking gets sampled.
For an agent, the practical rule is: low temperature wherever output must parse or be acted on (tool arguments, routing decisions, structured extraction), moderate where variety helps (candidate plans, drafts), and never treat it as a reliability control on its own.
Worked example with logits z = [2.0, 1.0, 0.1] for three candidate tokens (hypothetical values):
| T | p(token A) | p(token B) | p(token C) |
|---|---|---|---|
| 0.5 | 0.864 | 0.117 | 0.019 |
| 1.0 | 0.659 | 0.242 | 0.099 |
| 2.0 | 0.502 | 0.304 | 0.194 |
At T=0.5 the top token wins about 86% of draws. At T=2 it wins about half, and the tail takes 19%. Over a 300-token tool call, that tail mass compounds. One wrong field name at one position breaks the JSON.
Greedy decoding is the limit case. State two caveats. Greedy output is not bit-for-bit deterministic on GPU serving, because batching and floating-point reduction order vary. Providers also differ on whether T=0 is accepted and what the default is, so check the current API reference for the exact model you deploy.
Top-p and top-k truncate the distribution after temperature is applied. With temperature high and top-p unset, the tail is wide open. Tune one control at a time and log both.
Failure modes and how to catch them:
- Malformed tool calls at high T: invalid JSON or invented argument names. Validate every call against its schema and count failures per temperature setting.
- Looping at low T: the agent repeats the same failing action because a retry samples nearly the same distribution. Detect repeated identical tool-call payloads in traces. Fix it by feeding the error back into the prompt and capping retries, not by raising T blindly.
- Misleading evals: comparing runs at different temperatures and crediting the prompt change. Pin T and seed in the eval harness and report both.
Follow-up the interviewer is likely to ask: would you set T=0 everywhere? No. A fully greedy agent gets stuck in loops and cannot explore alternatives during planning. Set temperature per step, record it in the trace, and review it when a step's failure rate changes.
Curated: · Written: · Reviewed:
Q&A-6How can we ensure the LLM outputs data in a fixed format (JSON / CSV)?
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
A prompt that says "respond only in JSON" gets valid JSON most of the time. To get it every time, make the format a constraint that the decoder or the API enforces, validate the output in code anyway, and treat a parse failure as a counted, retryable event.
First I'd ask who consumes the output. If it's a downstream system (an ERP, a ledger, a claims engine), the schema is a contract, and I'd target JSON, not CSV. CSV has no types, no nesting, and quoting rules that models break on multi-line or comma-containing fields. If a client insists on CSV, the model emits JSON and a deterministic function writes the CSV.
Three layers:
- Constrained generation. OpenAI Structured Outputs with
response_formatset tojson_schemaandstrict: true(supported ongpt-4o-2024-08-06and later) restricts sampling to the schema. On Anthropic, define a tool with aninput_schemaand force it withtool_choice. Self-hosted, use vLLM guided decoding or llama.cpp GBNF grammars. Check each vendor's current docs for supported schema keywords, since strict modes reject some constructs. - One schema in code. A Pydantic v2 model is the single source of truth. Generate the JSON Schema from it, and validate with it.
- Validate, then retry with the error.
from typing import Literal
from pydantic import BaseModel, Field, ValidationError
class Invoice(BaseModel):
invoice_no: str
total: float = Field(ge=0)
currency: Literal["INR", "USD"]
def extract(text, llm, max_attempts=3):
feedback = None
for attempt in range(1, max_attempts + 1):
resp = llm.generate(text, schema=Invoice.model_json_schema(), feedback=feedback)
if resp.stop_reason == "max_tokens":
feedback = "output truncated"
continue
try:
return Invoice.model_validate_json(resp.text)
except ValidationError as e:
feedback = str(e)
raise ExtractionFailed(text_hash=hash(text), last_error=feedback)
One retry with the validator's message fixes most failures. Three attempts caps cost. Beyond that, route to human review rather than looping.
| Failure | What it looks like | Detection | Fix |
|---|---|---|---|
| Truncation | JSON cut mid-object | stop_reason == max_tokens | raise the limit, split the input |
| Number as string | "1,250.00" | parse-error rate per field | numeric type in schema, normalise in code |
| Invented enum value | "INR." | validation error | enum in schema, never free text |
| Refusal or empty content | no parseable payload | refusal and empty rate | explicit handling path |
| Drift after a model upgrade | parse failures rise overnight | fixed eval set run on every change | pin the model version, gate upgrades on evals |
Log every failed attempt with the input hash and the validator message. The failure distribution tells you whether to fix the schema, the prompt, or the model.
CSV is then mechanical: csv.writer with QUOTE_MINIMAL over validated objects, UTF-8, with the header written from the model's field order. Don't let the model write CSV rows directly.
When not to force structure: if the output is a free-text summary read by a person, a schema adds failure modes and buys nothing. Validate only what a machine will parse.
Curated: · Written: · Reviewed:
Q&A-7What is function calling in LLMs?
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
Function calling is a contract. Instead of answering in prose, the model emits a structured request: a tool name plus arguments that match a JSON Schema I declared. The model does not execute anything. My application validates the call, runs the function, and returns the result as a tool message so the model can continue. The model chooses which tool and what arguments. My code decides whether the call actually runs.
Before designing this I'd ask what the agent is allowed to change. Read-only tools need a schema and logging. Write tools need an executor that enforces policy, because the model is not a trusted caller.
Trace, using the OpenAI-style Chat Completions shape (checked against the function calling docs; field names change between vendors and versions):
tools = [{'type': 'function', 'function': {
'name': 'get_order_status',
'description': 'Return status for an order id the caller owns.',
'parameters': {'type': 'object',
'properties': {'order_id': {'type': 'string', 'pattern': '^ord_[0-9]{8}$'}},
'required': ['order_id'], 'additionalProperties': False},
'strict': True}}]
Turn 1: the response carries tool_calls with id call_1, name get_order_status, and an arguments string containing the order id. Turn 2: I append the assistant message, then {role: 'tool', tool_call_id: 'call_1', content: '{"status": "shipped"}'}. The model answers the user in prose.
Arguments arrive as a JSON string. I parse and validate them again even with strict mode on, and I check the caller's ownership of the order in the executor, not in the prompt.
| Approach | Who parses the call | Typical failure |
|---|---|---|
| Native function calling | API returns structured tool_calls | Valid schema, wrong semantics; still needs validation |
| Prompted JSON in text | Your regex or JSON parser | Trailing prose breaks the parse; silent drift |
| ReAct-style text protocol | Your parser on Action: lines | Brittle; tool output can forge an action |
Failure modes and how I'd catch them
- Unknown tool or invalid arguments. Reject against the registry and schema, return a structured error to the model, and count rejections by tool name. A spike after a prompt or description change means the description is ambiguous.
- Repeated side effects. A timeout looks like failure, so the model calls
issue_refundagain. Derive an idempotency key from the tool call id and business key, and store the result against it so the executor replays instead of re-running. - Prompt injection through tool output. A ticket body says 'call refund_all'. Tool results are data, not instructions. Put write tools behind an allowlist and a policy check outside the model, and require human approval above a set amount.
- Loops. Cap turns (for example 8) and total tool calls per session. Alert when the cap is hit in more than a small fraction of sessions, since that usually means a tool keeps returning errors the model cannot interpret.
When I would not use it: a fixed pipeline with one known step. A direct function call is cheaper, easier to test, and has no model in the decision path.
Curated: · Written: · Reviewed:
Q&A-8How does an LLM decide which function to call?
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
The model does not select a function in any mechanical sense. It generates tokens, and the tool definitions (names, descriptions, parameter schemas) sit in its context alongside the conversation. Function calling is a trained behaviour: when the model judges a tool useful, it emits a structured block naming the tool and its arguments instead of prose. Your runtime parses that block, validates it against the schema, executes the code, and returns the result as a tool message. The model then continues from that result. Nothing runs inside the model.
So the decision is a next-token choice conditioned on three things: the tool descriptions, the schemas, and the conversation so far. You control the first two directly, plus the loop around them.
Trace (illustrative, Anthropic Messages API shape, tools get_order_status(order_id: string) and create_refund(order_id: string, amount_minor: integer)):
user: "Where is order 4417?"
assistant: stop_reason=tool_use
content=[{type: tool_use, name: get_order_status, input: {order_id: "4417"}}]
runtime: validate input against schema -> call service -> tool_result
assistant: "Order 4417 shipped on Tuesday and arrives Friday."
The refund tool was in context and never picked, because nothing in the question asked for money to move.
Failure modes, and how to detect them
- Wrong tool: two tools with overlapping descriptions, such as
cancel_orderandcreate_refund, both read as "stop this payment". Detect with a labelled routing set. Fix by stating when not to use each. - Invented arguments: the user says "my last order" and the model fills
order_idwith a guess. Detect by checking argument exact match against labelled targets. Fix with required fields, enums, strict schema mode, and an instruction to ask the user when a required value is missing. - Spurious calls: the model calls a tool for a question it could answer directly. Detect with a set of no-tool questions and measure the false-call rate.
- Duplicate side effects: parallel or retried calls issue two refunds. Make side-effecting tools idempotent with a key the runtime generates, not the model.
Controls
| Lever | What it changes | What to measure |
|---|---|---|
| tool description | which tool is chosen | tool-selection accuracy |
| schema, strict mode | argument validity | schema-violation rate |
tool_choice (auto, required, none, named) | whether a call is forced | calls per turn |
| iteration cap | loop bound | runs hitting the cap |
Use required or a named tool when a step has one known tool. Use auto elsewhere. When the tool set is large, a routing classifier ahead of the agent is easier to test, but then the classifier is the decision and needs its own evaluation.
Report tool-selection accuracy, argument exact match and false-call rate separately on a labelled set of a few hundred utterances. A single aggregate accuracy hides the failure that matters most here, a wrong refund, which can be rare and still expensive.
Curated: · Written: · Reviewed:
Q&A-9How will you evaluate the retrieved chunks before generation?
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
I score each chunk against the question with a cross-encoder reranker, then check whether the surviving set can answer the question at all. These are two different checks. Relevance asks whether each chunk is on topic. Sufficiency asks whether the chunks together contain the facts the answer needs. Vector similarity handles the first loosely and the second not at all.
Retrieval returns the top 50 chunks by approximate nearest neighbour search. A cross-encoder reads each (query, chunk) pair together, which is more accurate than comparing two independent embeddings. I keep the top 5 that clear a threshold tau, and if nothing clears it, the system abstains instead of generating.
| Check | Mechanism | What it catches |
|---|---|---|
| Freshness | filter on effective_date and superseded metadata before scoring | an old policy that still ranks high |
| Relevance | cross-encoder score >= tau | topical match with the wrong entity, site or year |
| Redundancy | drop a chunk with cosine > 0.95 to one already kept | five copies of one paragraph crowding out a second source |
| Sufficiency | check that every entity the question names appears in the kept set | a question needing two documents, where only one was retrieved |
Trace (hypothetical figures). Query: "What is the notice period for contractors at the Pune site?" Chunk 31, a 2023 contractor policy marked superseded, is removed by the freshness filter. Of the rest, chunk 04 ("Contractor notice period: 14 days, Pune, effective 2024-04-01") scores 0.88, chunk 17 ("Notice period: 30 days, permanent staff") scores 0.31 and falls below tau = 0.5. Sufficiency passes because one kept chunk names contractors and Pune. The generator gets chunk 04 alone, with its citation.
Measuring it. I label a few hundred real queries with their gold chunks offline and track context precision@5, recall@50, MRR and nDCG@10 for every release. The metrics point to different fixes. Falling recall@50 means retrieval or chunking is broken. Falling precision@5 with stable recall means the reranker or threshold is wrong. Online, I watch the abstention rate and the share of answers citing a chunk that scored below tau.
Failure modes. A threshold tuned on one embedding or reranker model stops meaning anything after a model swap, because scores are not comparable across models. I recalibrate tau on the labelled set whenever either model changes. Faithfulness is a generation metric, so it does not replace these checks. A faithful answer built from the wrong chunk still fails the user.
Curated: · Written: · Reviewed:
Q&A-10Which evaluation metrics will you use (e.g., Recall@K, Precision@K)?
Reported in TCS GenAI ML Interview Experience and Questions - LinkedIn
The 90-second answer
I split evaluation into the parts that can fail independently: retrieval, tool use, and the end-to-end task. Recall@K and Precision@K belong to retrieval, because the agent can only reason over evidence it was handed. I'd report both at a fixed K, then add rank-aware metrics and task success so a good retriever can't hide a bad agent.
Definitions, with a hypothetical example. For one query with 10 labelled relevant chunks, the retriever returns 5 chunks and 3 are relevant:
- Precision@5 = 3 / 5 = 0.60
- Recall@5 = 3 / 10 = 0.30
Recall@K is the one I watch for agents, since a missed clause in a contract summary is an unrecoverable miss. Precision@K matters because every irrelevant chunk spends context budget and can pull the model off course.
def precision_recall_at_k(ranked_ids, relevant_ids, k):
top = ranked_ids[:k]
hits = sum(1 for d in top if d in relevant_ids)
return hits / k, hits / len(relevant_ids)
| Metric | Question it answers | When I use it |
|---|---|---|
| Recall@K | Did the evidence we need make it into the top K? | Retrieval gate before generation |
| Precision@K | How much of the context is noise? | Context budget and latency tuning |
| MRR | How early does the first relevant hit appear? | Single-answer lookups |
| nDCG@K | Are graded relevance labels ordered well? | Ranking changes between rerankers |
| Task success (pass@1, pass^k) | Did the agent finish correctly, and reliably across k runs? | End-to-end release decision |
| Tool-call accuracy | Right tool, right arguments, no extra calls | Agent loop regressions |
Two things decide whether these numbers mean anything. First, the label set. I'd want around 200 queries drawn from real traffic, relevance judged by two annotators, and agreement reported (Cohen's kappa). Recall@K is only as honest as the judgments: unlabelled relevant chunks count as misses. Second, the K. Choose it from the context budget before looking at results, or the metric is tuned to the test set.
Failure modes I'd check for:
- Queries with zero relevant items silently dropped from the mean, which inflates the score.
- Test queries overlapping with documents used to tune chunking or prompts (leakage).
- Offline Recall@K rising while online task success stays flat. That usually means the retriever is returning the right document but the agent ignores the relevant passage. I'd check that with a trace of which chunks the model cited.
For a TCS engagement with regulated client documents, I'd report every metric per tenant and per document class, not one global average. A 0.92 aggregate can hide a 0.55 on the scanned-PDF slice, which is the slice the client will complain about.
Curated: · Written: · Reviewed:
Q&A-11Write a RAG implementation in Python.
Reported in Tata Consultancy Services AI Engineer Interview Guide 2026
The 90-second answer
I would build the smallest loop that works first: chunk the documents, embed the chunks, store the vectors, retrieve the top-k by cosine similarity, put the chunks into a prompt with source ids, and generate. I keep the embedder and the LLM behind plain callables so the pipeline can be tested without a vendor. Assumptions I would state out loud: the text is already extracted from the source files, and the corpus fits in memory, so a flat numpy scan is acceptable up to a few hundred thousand chunks.
Checked against Python 3.11 and NumPy 1.26:
from dataclasses import dataclass
from typing import Callable
import numpy as np
def chunk_text(doc_id: str, text: str, size: int = 800, overlap: int = 100):
assert 0 <= overlap < size
step = size - overlap
return [(doc_id, text[i:i + size]) for i in range(0, max(len(text) - overlap, 1), step)]
@dataclass
class Chunk:
doc_id: str
text: str
vec: np.ndarray
class RAG:
def __init__(self, embed: Callable[[list], np.ndarray], llm: Callable[[str], str], k: int = 4):
self.embed, self.llm, self.k = embed, llm, k
self.index: list[Chunk] = []
self.matrix = np.empty((0, 0))
def ingest(self, docs: dict):
pieces = [p for doc_id, text in docs.items() for p in chunk_text(doc_id, text)]
vecs = np.asarray(self.embed([t for _, t in pieces]), dtype=np.float32)
vecs /= np.linalg.norm(vecs, axis=1, keepdims=True)
self.index = [Chunk(d, t, v) for (d, t), v in zip(pieces, vecs)]
self.matrix = vecs
def retrieve(self, question: str, allowed_docs: set):
mask = np.array([c.doc_id in allowed_docs for c in self.index])
q = np.asarray(self.embed([question])[0], dtype=np.float32)
scores = self.matrix @ (q / np.linalg.norm(q))
scores = np.where(mask, scores, -np.inf)
top = np.argsort(-scores)[: self.k]
return [(self.index[i], float(scores[i])) for i in top if mask[i]]
def answer(self, question: str, allowed_docs: set):
hits = self.retrieve(question, allowed_docs)
if not hits:
return 'No accessible source matches this question.', []
context = '\n\n'.join(f'[{n}] ({c.doc_id}) {c.text}' for n, (c, _) in enumerate(hits, 1))
prompt = ('Answer only from the sources. Cite them as [n]. '
'If the sources do not contain the answer, say so.\n\n'
f'Sources:\n{context}\n\nQuestion: {question}\nAnswer:')
return self.llm(prompt), [c.doc_id for c, _ in hits]
In the interview I would talk through three decisions while typing. The access filter runs before scoring, so a restricted document can never reach the prompt. Chunks carry their doc_id all the way to the answer, which makes citations checkable. Re-ingesting a document means dropping its old chunks, not appending.
Trace (hypothetical figures)
Corpus: 3 policy documents, 41 chunks. Question: 'What is the refund window for annual plans?' with allowed_docs={'policy_v2', 'faq'}.
| rank | doc | score |
|---|---|---|
| 1 | policy_v2 | 0.81 |
| 2 | policy_v2 | 0.74 |
| 3 | faq | 0.52 |
| 4 | faq | 0.49 |
The policy_v1 chunk that scored 0.88 is never considered because it fails the mask. That is the behaviour I want, and it is the first thing I would test.
Failure modes and how I detect them
- Answers cite
[5]when k is 4. Validate every citation number against the number of chunks passed in. - A policy changes and the answer quotes the old one. Track a content hash per document and re-ingest on change. Without it the index silently goes stale.
- Relevant text is split across a chunk boundary. The 100-character overlap helps only partly. Measure recall@k on a labelled question set before tuning anything else.
- Exact identifiers such as error codes or contract numbers rank poorly under embeddings. Add BM25 and merge the rankings.
When not to use this
Above roughly a million vectors, or with concurrent writers, move the index into a vector store with an approximate index such as HNSW. The class boundary stays the same; only retrieve changes.
Curated: · Written: · Reviewed:
Q&A-12What is Agentic AI? Explain the concept and provide a real-world example where Agentic AI can be effectively applied.
Reported in Tata Consultancy Services (TCS) Prime Interview Experience | On-Campus (2026) - GeeksforGeeks
The 60-second answer
Agentic AI is a system where a language model runs a loop: it reads a goal, chooses the next action, calls a tool, reads the result, and repeats until a stop condition is met. A chatbot answers once. A RAG call retrieves once and answers once. An agent decides the sequence of steps at runtime, so the developer defines the tools, the permissions and the stop rules, and the model defines the path through them. The engineering work sits in what the loop can touch, how it stops, and how you prove what it did.
Before designing anything I would ask whether the agent is read-only or can write to systems of record. That answer sets the permission model and how much human approval the loop needs.
A worked example: claims triage at a general insurer (hypothetical figures)
A motor claim arrives with photos and a free-text description. The agent gathers facts, estimates the damage band and drafts a settlement for an adjuster to approve.
def run_agent(goal, tools, max_steps=8, token_budget=60_000):
history = [system_prompt(), user(goal)]
spent = 0
for _ in range(max_steps):
msg = llm.complete(history, tools=tools.schemas())
spent += msg.usage.total_tokens
if spent > token_budget:
raise BudgetExceeded(run_id, spent)
if not msg.tool_calls:
return msg.content
for call in msg.tool_calls:
tool = tools.get(call.name)
if tool.has_side_effect and not approvals.granted(call):
result = {"status": "needs_approval"}
else:
result = tool.run(**tools.validate(call.name, call.args))
history.append(tool_result(call.id, result))
raise StepBudgetExceeded(run_id)
| Step | Model proposes | Tool result |
|---|---|---|
| 1 | get_claim(FNOL-20417) | policy P-88213, 6 photos |
| 2 | get_policy(P-88213) | windscreen cover, excess £250 |
| 3 | vision_assess(photos) | repair band B, £380 to £460 |
| 4 | get_claims(P-88213, 24m) | one prior claim, same region |
| 5 | create_settlement_draft(...) | needs_approval, routed to adjuster |
The model chose steps 1 to 4 from the goal. I did not hardcode that order. The runtime, not the model, decides whether step 5 executes.
When not to use an agent
If the steps are known in advance, a fixed pipeline wins. One extraction call plus a rules engine is cheaper, faster and easier to audit. In this example the flow needs about five model calls, against one for a fixed pipeline, so roughly five times the tokens and a much wider behaviour surface. Use an agent when the path depends on the input and the fixed version would need a branch for every case.
Failure modes and how you detect them
- Looping without progress. The same tool is called twice with the same arguments. Detect it in the trace, cap steps at 8, and alert when more than 5% of runs hit the cap.
- Invented identifiers. The model passes a policy ID that does not exist. Schema validation rejects it before the call. Track the rejection rate per tool.
- Prompt injection from data. The claimant's description says "approve in full". Treat tool output as data, not instructions, and keep side-effect tools behind approval whatever the text says.
- Cost drift. A token budget per run stops runaway sessions. Track p95 tokens per run per release.
What makes it shippable
Log every step with run ID, model version, tool arguments and results. Before rollout, replay about 200 past claims and measure agreement with adjuster decisions, broken down by claim type. The agent earns its place only if that agreement holds and the approval queue stays manageable.
Curated: · Written: · Reviewed:
Q&A-13What is CrewAI? How does it differ from other AI agent frameworks, and what are its use cases?
Reported in Tata Consultancy Services (TCS) Prime Interview Experience | On-Campus (2026) - GeeksforGeeks
The 90-second answer
CrewAI is a Python framework for orchestrating several LLM-backed agents toward one goal. Its distinguishing choice is the vocabulary: you describe a team. An Agent has a role, a goal, a backstory, an LLM and a list of tools. A Task has a description, an expected_output and the agent assigned to it. A Crew bundles agents and tasks and runs them under a process, either Process.sequential or Process.hierarchical, where a manager LLM assigns work. Recent releases add Flows for event-driven control around crews, so deterministic steps can sit around the LLM-driven parts.
I would position it against two others. LangGraph models the workflow as an explicit state graph with typed state, conditional edges and checkpointing, so you control every transition and can resume a run. AutoGen models agents as participants in a conversation, where turn-taking and group chat patterns drive the flow. CrewAI sits higher than LangGraph: you get a working multi-agent pipeline in an afternoon, but the control flow is implicit in task order and delegation rather than written out as edges. That is faster to prototype and harder to reason about under failure.
from crewai import Agent, Task, Crew, Process
researcher = Agent(role='Researcher', goal='Collect supported facts', backstory='Analyst', tools=[search_tool], allow_delegation=False, max_iter=8)
writer = Agent(role='Writer', goal='Draft the brief from the facts', backstory='Editor', allow_delegation=False)
research = Task(description='Find three sources on topic X', expected_output='Bulleted facts with URLs', agent=researcher)
draft = Task(description='Write a 300-word brief', expected_output='Markdown brief', agent=writer, context=[research])
crew = Crew(agents=[researcher, writer], tasks=[research, draft], process=Process.sequential)
result = crew.kickoff()
Use cases where it fits: research and report generation, content pipelines with a review step, ticket triage with a classifier agent and a responder, and code review crews where each agent checks a different concern. It fits badly where the path must be auditable step by step or where latency and cost per run are tightly capped.
Failure modes I would check first:
- Runaway loops. Without
max_iterandmax_rpmcaps, an agent retries a failing tool until token spend explodes. Set both, and log iteration counts per task. - Hierarchical manager drift. The manager LLM can misroute or re-run tasks. Each extra manager call adds tokens and latency, so measure cost per completed crew before choosing hierarchical.
- Unbounded delegation.
allow_delegation=Truelets agents pass work to each other, which hides the path. Turn it off unless the task needs it. - Nondeterministic output that breaks downstream parsing. Use
output_pydanticor a structured expected output, and validate it in code.
Detection is the same for all of them: trace each task's inputs, outputs, tool calls and token counts, then compare pass rates on a fixed evaluation set before and after any change to prompts or process type.
My recommendation: start with CrewAI when the work is a bounded set of roles with a clear hand-off order, and move to an explicit graph once you need checkpoints, branching you must audit, or strict cost ceilings.
Curated: · Written: · Reviewed:
Q&A-14How would you use AI to analyze the data and recommend strategies to increase the company's revenue?
Reported in Tata Consultancy Services (TCS) Prime Interview Experience | On-Campus (2026) - GeeksforGeeks
The 90-second answer
I'd build this as a bounded analysis agent rather than a chatbot that guesses at revenue. The model plans, calls read-only tools over governed data, and returns a recommendation with the query behind every number. A person approves anything that touches customers.
First I'd pin the question. "Increase revenue" can't be tested, so I'd ask which revenue line (new logos, expansion, renewal, pricing), which time window, and what the baseline is. Then the agent decomposes revenue into price, volume, conversion and retention, and finds which term has the largest gap against target or prior year.
The tools are narrow: a SQL tool that can only query certified views, a metric catalogue so "revenue" means the finance definition, a segmentation tool, and a read on the experiment registry. No write access, and no raw PII columns.
| Step | What the agent does | Guardrail |
|---|---|---|
| Scope | Resolves metric definitions from the catalogue | Rejects unknown metric names |
| Decompose | Splits revenue into price, volume, conversion, retention | Components must reconcile to the finance total within 0.5% |
| Segment | Finds cohorts with an outsized gap | Minimum cohort size, e.g. n ≥ 200 |
| Hypothesise | Proposes 2 to 3 levers with expected lift | Each lever cites a query id |
| Validate | Checks prior experiments and holdouts | Lift is stated as a range, not a point |
Worked trace (hypothetical figures). Mid-market renewal revenue fell 6% quarter on quarter. The agent finds 1,200 lapsed accounts. 70% of them sit in the 40-seat tier with no business review in the last 90 days, and that group churns at roughly three times the rate of the rest. Average monthly revenue per account is ₹4,500. Suppose the outreach team contacts the 840 accounts without a recent review and cuts their churn by 6 points: 840 × 0.06 ≈ 50 accounts retained, 50 × ₹4,500 = ₹2.25 lakh per month, about ₹27 lakh annualised. The agent reports that as a range and recommends a 20% randomised holdout, because "no review" may be a symptom of account health rather than a cause of churn.
Failure modes and detection
- Correlation read as causation. Detected only by a holdout. If predicted lift and measured lift diverge beyond the stated range, the recommendation is flagged, not re-scored.
- Stale or mis-joined data. The reconciliation check fails the run, and the agent returns no recommendation.
- Invented figures. The output schema rejects any number without a query id.
- Spurious segments. Test 40 segments at p < 0.05 and about two will look significant by chance. Correct for multiple comparisons, or pre-register the segments before looking.
When not to use this. For pricing elasticity with no historical variation, a model can't identify the answer. Run a price test. An agent can propose the test, but it can't substitute for one.
The output is a structured record: lever, expected lift range, evidence query ids, confidence, owner, and the experiment that would confirm it.
Curated: · Written: · Reviewed:
Q&A-15Explain clustering, regression and RAG.
Reported in Tata Consultancy Services AI Engineer Interview Guide 2026
The 90-second answer
These are three different kinds of problem, so I'd separate them first. Clustering and regression are classical machine learning on structured data: clustering groups unlabelled rows by similarity, regression predicts a continuous number from labelled examples. RAG is a generation pipeline: it retrieves relevant text at query time and gives it to an LLM so the answer is grounded in documents you control. In an agentic system, clustering might group incoming support tickets for routing, regression might forecast resolution time for SLA planning, and RAG is usually the tool the agent calls when it needs policy or product text.
| Clustering | Regression | RAG | |
|---|---|---|---|
| Learns from | unlabelled features | labelled (x, y) pairs | documents plus a query |
| Output | cluster id per row | a number per row | generated text with citations |
| Typical model | k-means, HDBSCAN | linear, gradient boosting | embedding model + LLM |
| Main metric | silhouette, stability across seeds | RMSE or MAE vs a naive baseline | recall@k on labelled queries, faithfulness |
Clustering. K-means minimises within-cluster squared distance. You must pick k, and it assumes roughly spherical clusters on comparable scales:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(X_raw)
labels = KMeans(n_clusters=6, n_init=10, random_state=0).fit_predict(X)
Without scaling, the feature with the largest numeric range dominates the distance. Check stability by rerunning with different seeds. If the clusters move, they are not telling you much.
Regression. Start with a baseline that predicts the mean, then fit LinearRegression or a boosted tree. Report error in the target's units. The usual failure is leakage: a feature that is only known after the outcome, such as a close timestamp used to predict resolution time, produces an excellent offline score and a useless model in production.
RAG. Ingest documents, split them into chunks, embed and index them, then at query time retrieve the top k chunks, build a prompt that includes them, and generate an answer that cites chunk ids. The failures sit mostly in retrieval, not generation. A wrong chunk size splits the answer across chunks, a stale index answers from last quarter's policy, and a model that is never told to refuse fills gaps from its training data. You detect these by measuring recall@k against a labelled query set before you touch the prompt.
What I'd say in the room: I'd not treat these as interchangeable. I'd ask what the label is, what error is expensive, and whether the answer must be traceable. Those answers pick the method.
Curated: · Written: · Reviewed:
Q&A-16How would you configure an LLM?
Reported in Tata Consultancy Services AI Engineer Interview Guide 2026
The 90-second answer
I'd treat configuration as a versioned artefact that gets reviewed and deployed, not as constructor arguments scattered through the code. Before touching any parameter I'd settle three things: what the model must do and how fast, what data it will see and its classification, and where inference has to run. On a TCS client engagement that usually means inference inside the client's own tenancy (Azure OpenAI, Amazon Bedrock, or a self-hosted open-weight model), so residency and logging rules come first.
Then I'd pin everything the output depends on: the exact model snapshot, the system prompt by version, token limits, timeouts and retry policy. A floating alias such as latest means behaviour can change without a deploy.
llm:
provider: azure_openai
deployment: triage-prod # points at a pinned model snapshot, never an alias
api_version: <pinned in platform config>
timeout_s: 30
max_retries: 2 # retry only 429 and 5xx, with jittered backoff
max_output_tokens: 1024
temperature: 0.2
system_prompt_version: triage-v7
max_agent_steps: 8
tools: [lookup_ticket, create_ticket]
logging: {store_prompts: false, redact: [pan, aadhaar, email]}
| Setting | What it controls | How I set it |
|---|---|---|
| deployment / snapshot | Which weights answer | Pinned; changed only through eval gate |
| max_output_tokens | Truncation and cost per call | Sized from p95 output, checked against finish_reason == "length" |
| context budget | Prompt plus output must fit the window | Counted per request, retrieval trimmed first |
| timeout_s, max_retries | Tail latency and duplicate spend | Retries only on 429/5xx; never retry a tool call with side effects blindly |
| max_agent_steps | Runaway tool loops | Hard cap, returns a partial result with a reason |
| logging / redaction | Data exposure | Prompts not stored by default; PII redacted before logs |
Sampling parameters like temperature and top-p matter, but they are the last thing I reach for. If answers are wrong, the fix is usually retrieval, the prompt or the tool schema, not a lower temperature.
The failure modes I'd name in the design review:
- Silent model change. A vendor rolls a model forward and scores drop. Detection: a golden eval set runs in CI and nightly against the pinned deployment, with a threshold on task accuracy and format-validity rate.
- Truncated output.
max_output_tokenstoo low produces half a JSON object. Detection: alert on thelengthfinish reason and count schema-validation failures per route. - Cost blowup. An agent loop with no step cap burns budget on one bad ticket. Detection: per-request token and step metrics with a per-tenant spend alert.
- Injection through tool output. Retrieved text tells the agent to call
create_ticketwith someone else's account. Mitigation: tool arguments are validated server-side against the caller's permissions, not trusted from the model.
So the answer I'd give a panel is: pin the model, budget the context, bound the loop, log without storing sensitive content, and gate every change to any of them on the eval suite.
Curated: · Written: · Reviewed:
Q&A-17Name five LLMs.
Reported in TCS Digital Interview Experience – Chennai
The 90-second answer
Five I would name, with the access route, because on a TCS client engagement the route often decides whether a model is usable at all:
| Model | Vendor | Weights | Context window (documented) | Typical access route |
|---|---|---|---|---|
| GPT-4o | OpenAI | closed | 128K tokens | OpenAI API, Azure OpenAI |
| Claude 3.5 Sonnet | Anthropic | closed | 200K tokens | Anthropic API, AWS Bedrock, Google Vertex AI |
| Gemini 1.5 Pro | closed | up to 2M tokens | Gemini API, Vertex AI | |
| Llama 3.1 70B / 405B | Meta | open weights, community licence | 128K tokens | self-hosted, Bedrock, Azure, Vertex |
| Mistral Large 2 | Mistral AI | weights under a research licence, commercial use needs a separate licence | 128K tokens | Mistral API, Azure, self-hosted |
Versions and limits are as of my last check. I would confirm them against each vendor's model card before quoting them to a client, because model names, context sizes and deprecation dates change within a few months.
The list is the easy half. The follow-up is usually why one of them fits a workload, so I would frame it as three questions:
- Data boundary. Can the client's data leave their tenancy? If not, the choice narrows to self-hosted open weights or a cloud-hosted model inside the client's own cloud account.
- Task fit. For extraction and classification, a smaller model with good structured-output support is often enough and costs less per call. For long multi-step agent plans, I would test the larger models on our own traces before deciding.
- Operational cost. Compare cost per successful task, not cost per token. A cheaper model that needs two retries and a human review step can cost more.
Two failure modes I would raise unprompted. First, an alias such as latest can change behaviour under the agent without a deploy, so I pin dated snapshot identifiers and treat an upgrade as a release with its own evaluation run. Second, a model that passes a demo can regress on the client's documents, so the eval set needs real retrieved chunks and tool-call traces, with pass rates tracked per model version.
Curated: · Written: · Reviewed:
Q&A-18Write Fibonacci in Python.
Reported in TCS Digital Interview Experience – Chennai
The 90-second answer
I'd write it iteratively. Two variables carry the last pair of values, so it runs in O(n) time and O(1) extra space, and it never hits the recursion limit. I'd confirm the input contract first: is n a non-negative int, and is it zero-indexed with F(0) = 0? I'll assume both unless you say otherwise.
def fib(n: int) -> int:
if n < 0:
raise ValueError('n must be >= 0')
a, b = 0, 1
for _ in range(n):
a, b = b, a + b
return a
Trace for n = 5. The pair (a, b) moves through (0,1) → (1,1) → (1,2) → (2,3) → (3,5) → (5,8). The loop runs five times, and a is 5, which is F(5).
Why not the textbook recursion. return fib(n-1) + fib(n-2) is correct but makes 2·F(n+1) − 1 calls. For n = 30 that is 2,692,537 calls, and each call repeats work already done. Running time is exponential, about φⁿ.
Memoised alternative. functools.cache (Python 3.9+) turns the same recurrence into O(n) time:
from functools import cache
@cache
def fib(n: int) -> int:
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
The failure mode is depth. Each call waits on fib(n-1), so the stack grows to n frames. CPython's default recursion limit is 1000, so fib(2000) raises RecursionError even though the cache makes it fast. The iterative version has no such limit.
Big integers. Python ints are arbitrary precision, so fib(1000) is exact. Fixed-width languages overflow at F(93) for a signed 64-bit int, and that is the first thing I'd check if the interviewer asks for Java or C.
Where this matters for agents. I wouldn't have an LLM compute Fibonacci inside an agent loop. I'd expose the function as a deterministic tool with a typed signature and unit tests, so the model only decides when to call it and never produces the digits itself.
Tests I'd add. fib(0) == 0, fib(1) == 1, fib(10) == 55, fib(-1) raises, and a cross-check of the iterative and memoised versions for n in 0..200.
Curated: · Written: · Reviewed:
Curated: · Written: · Reviewed: