Semantic search over your notes

What you'll build

Step-by-step build

1. 1

   Set up the project and the index

   Scaffold a Next.js app and create a free MongoDB Atlas M0 cluster. Add a vector search index on your passages collection with 384 dimensions and cosine similarity, matching all-MiniLM-L6-v2. Load the embedding model in a Node API route so you never send note text to a third party.

2. 2

   Read and chunk the notes

   Walk the target folder and read every .md and .txt file. Use gray-matter to strip and keep frontmatter, then split the body into overlapping chunks of roughly 300 to 500 tokens with about 50 tokens of overlap so a sentence is never cut in half between chunks. Keep the filename and nearest heading with each chunk.

3. 3

   Embed and store

   For each chunk, generate its embedding and insert a document like { text, embedding, source, heading, mtime } into MongoDB. Batch the inserts so indexing a few hundred notes takes seconds, not minutes. Store the file's modified time so you can skip unchanged files on the next run.

4. 4

   Build the query path

   On a search, embed the query string with the exact same model, then run an Atlas $vectorSearch aggregation to get the top 10 nearest chunks. Return the text, score, source, and heading for each. Confirm that semantically similar but differently worded queries surface the right note.

5. 5

   Render results well

   Show each hit as a card with the filename, heading, similarity score, and a short snippet. Make the filename a link that opens the full note. A visible score helps you feel where the model draws the line between relevant and not.

6. 6

   Make re-indexing incremental

   On startup or a manual 'reindex' button, compare each file's modified time against what you stored and only re-embed the files that changed. Delete chunks for files that no longer exist. This keeps a growing note collection fast to update.

7. 7

   Add the optional answer box

   For the stretch goal, take the top 3 passages, put them in a prompt to Groq, and ask for a two-sentence answer that cites the source filenames. Keep the raw results visible underneath so the user can always verify the summary against the actual notes.

8. 8

   Polish and ship

   Add an empty state, a loading indicator, and a keyboard shortcut to focus the search box. Write a README that explains how to point the app at your own folder. Deploy and confirm search works on the live URL with your real notes.

Done when

• ✓You search for an idea using words that appear nowhere in the target note and it still comes back in the top results.
• ✓Results are ordered so the most relevant passage is first, and the scores make that ordering believable.
• ✓Editing one note and re-indexing updates only that note's chunks, not the whole collection.
• ✓A folder you have never indexed before works end to end with no code changes.
• ✓The live URL loads and returns results for someone who is not you.

Ship it

Dockerize the app or deploy the Next.js project straight to Vercel on the free tier, pointing it at your Atlas M0 cluster. Ship a live URL plus a README that shows a couple of before-and-after searches where keyword search fails and semantic search wins. Include a short GIF of a query and its results so a recruiter grasps it in five seconds.

What it proves: You understand embeddings, chunking, and vector nearest-neighbor search end to end, the exact primitives that every RAG system is built on, and you can turn a folder of raw text into a working meaning-based search engine.

Paste this into Cursor, Claude, or ChatGPT and build it step by step.

You are my senior AI engineer pair. Help me build "Semantic search over your notes" step by step. Semantic search means matching by meaning using embeddings (vectors that place similar text close together) rather than keywords. Stack: Next.js (UI + API routes), all-MiniLM-L6-v2 for 384-dim embeddings running locally, MongoDB Atlas Vector Search (free M0) for storage, gray-matter for markdown parsing.

Requirements:
1. Point the app at a folder of .md/.txt files; chunk each into ~400-token overlapping passages keeping filename and heading.
2. Embed each passage and store { text, embedding, source, heading, mtime }.
3. On a query, embed it with the same model and run an Atlas $vectorSearch for the top 10 passages.
4. Render results as cards with source, heading, score, and snippet, linking to the full note.
5. Re-index only files whose modified time changed since last run.

Work in this order: scaffold the app and the Atlas vector index, then folder reading and chunking, then embedding and storage, then the query path, then the results UI, then incremental reindex. Give me the commands and code for each step and STOP after each so I can test. Do not write the whole app at once.