Skip to content
Multimodal AI Hackathon 2026 · Track D

Study smarter with an AI tutor that cites its sources

Upload your lectures, textbooks and slides. Get cited answers when your material has the answer — or a clearly separate general-knowledge response when it doesn’t. Then practice with quizzes that adapt to what you know.

Evaluation not run yet.

Open book connected to lesson video, notes and a glowing study idea
Cited sourceLecture 7 · p.42
Capabilities

Built around the six track requirements

Each capability below maps directly to a numbered requirement in the Track D problem statement.

Multimodal Knowledge Base

Ingest PDFs, slide decks, lecture video, audio and images with no manual preprocessing. Every unit keeps its exact origin.

Source Grounding

Answers grounded in your material cite the page, slide or timestamp. If the material does not cover it, a clearly labelled general-knowledge answer stays separate.

Adaptive Assessment

MCQ, short answer and numerical questions on the scope you choose, verified against your material and tagged to a topic.

Learner Model

Bayesian knowledge tracing plus a forgetting curve gives you a mastery estimate per topic that updates after every answer.

System Evaluation

Faithfulness, relevancy, context precision and recall measured against a test set with known source locations.

Course Map

A visual graph of topics and their prerequisites, colour-coded by your current mastery.

Revision Material

Flashcards and summaries point you back to the topics that could use another look.

Study Schedule

A plan built from weak topics, forgetting-curve decay and how many days are left before your exam.

Hindi / English

Ask in English or Hindi. Hindi explanations keep technical terms in English so nothing gets lost in translation.

Audio Tutoring

Dictate a question with your microphone and have answers read aloud.

How it works

Four steps from raw material to mastery

No manual tagging, no spreadsheet of topics. The pipeline builds the structure as it ingests.

  1. 01
    Clay-style document, image, notes, and video icons ready to upload

    Upload your material

    Drop in lecture PDFs, slide decks, recorded lectures or images. StudyMind extracts text, figures and speech with timestamps.

  2. 02
    Clay-style open book with connected nodes representing a topic map

    Knowledge base is built

    Content is chunked, embedded, and mapped to topics, subtopics and key concepts — each unit tagged back to its origin.

  3. 03
    Clay-style checklist and blank chat bubble representing a quiz or tutor chat

    Chat or take a quiz

    Ask anything and get a cited answer, or generate an adaptive assessment focused on what you have not mastered yet.

  4. 04
    Trophy and rising progress bars representing learning progress

    See what changed

    Your mastery updates after every answer. The dashboard shows what improved, what is fading, and what to do next.

System architecture

A retrieval pipeline with a single AI seam

Every AI call goes through one provider module, so the model can be swapped without touching business logic.

BrowserNext.js 14 · VercelExpress APITypeScript · ReplitMongoDB AtlasMongooseGemini APItext · vision · embedHTTPS + JWTIngestion workersPDF · PPTX · video → chunks

Browser → API

JSON over HTTPS with a bearer JWT on every protected route. CORS is locked to the deployed frontend origin.

Ingestion workers

Parse, chunk, embed and topic-tag each source. Unit-level failures are recorded without failing the whole source.

Provider seam

One interface supports Gemini-backed answers and an offline fallback for source-based study.

Grounding & trust

Trust is a mechanism, not a promise

Cited answers stay grounded, and general answers stay clearly separate.

Cited answers stay grounded; general answers stay separate

The tutor checks your uploads first. When they do not cover a question, any general-knowledge response is shown separately with a clear warning and no citations.

Citations are audited after the fact

Any citation pointing at a unit that was not actually retrieved is stripped before the answer reaches you, and the count of stripped citations is recorded in the evaluation metrics.

Example cited answerFully grounded

Why does the Clock algorithm exist?

The Clock or Second Chance algorithm approximates LRU by keeping a reference bit on each frame. On eviction the hand clears the reference bit and moves on; only a frame whose bit was already clear is actually evicted.

Lecture 7 · p.42Slides · 12:44

General knowledge — not from your uploaded material: Questions your uploads do not cover can still get a separate general answer.

Illustrative mastery curve

Illustrative values — your real curve is generated from your own answers.

Virtual memory86%
Cache hierarchy74%
Page faults52%
Deadlock41%
Atomics28%
weakdevelopingstrong
Adaptive learning

It knows what you know — and what you are forgetting

A per-topic mastery probability, updated after every answer, decayed over time by a forgetting curve.

Bayesian knowledge tracing. Each topic is tracked as a probability of being mastered. A correct answer pushes it up, a wrong one pulls it down — but a wrong answer after long success is treated as a slip, not as forgetting everything.

Forgetting curve. Retention decays exponentially with time. Topics you have not touched resurface on their own in the study plan.

Zone of proximal development. Quizzes deliberately target topics sitting between 30% and 70% mastery — where practice buys the most learning.

System evaluation

Measured, not asserted

Scored against a test set where every question has known source locations, plus deliberately off-material queries.

RAG pipeline metrics

Evaluation not run yet

MetricWhat it measuresScore
FaithfulnessAnswer claims supported by cited excerptsEvaluation not run yet
Answer relevancyDoes the answer address the question askedEvaluation not run yet
Context precisionRetrieved units that are actually relevantEvaluation not run yet
Context recallGround-truth units that were retrievedEvaluation not run yet
Citation accuracyCitations pointing at the correct unitEvaluation not run yet
Outside-material detectionQuestions correctly identified as outside your uploaded sourcesEvaluation not run yet

Tech stack

Next.js 14React 18TypeScriptTailwind CSSExpressNode.jsMongoDBMongooseGoogle GeminiVercelReplitRAGAS
FAQ

Questions students actually ask

PDF, PPTX, PNG and JPG, MP4 video, MP3 and WAV audio, plus plain text. PDFs are read page by page, slide decks keep slide numbers and speaker notes, and video is transcribed into timestamped segments. Every extracted unit remembers where it came from.

Start studying smarter

Upload your first lecture and see a cited answer in under a minute.