Meeting summarizer with action items

What you'll build

Step-by-step build

1. 1

   Handle the upload and audio prep

   Accept an audio or video upload and use ffmpeg to convert it to a standard 16kHz mono WAV. Normalizing the format up front makes Whisper faster and more accurate and avoids codec surprises later.

2. 2

   Transcribe with Whisper

   Run Whisper to produce a transcript with segment timestamps. Pick a model size that fits your free compute; a smaller model on CPU is fine for a demo. Keep the timestamps because you will use them for citations and the stretch goals.

3. 3

   Chunk long transcripts

   A long meeting will exceed the model's context window, so split the transcript into chunks on natural boundaries like pauses or speaker turns, keeping each comfortably under the limit. Preserve timestamps across the split so nothing loses its place in time.

4. 4

   Summarize faithfully

   For long meetings, summarize each chunk, then summarize the summaries into one final overview. Prompt the model to stick to what the transcript says and not to invent outcomes. A faithful, boring summary beats an impressive fabricated one.

5. 5

   Extract structured action items

   In a separate call, ask the model to return decisions and action items as JSON, each item shaped { task, owner, dueDate }, using null when an owner or date was not stated rather than guessing. Never assign an owner the meeting did not name.

6. 6

   Validate and merge

   Validate the action-item JSON against your schema and combine it with the summary into one result object. If validation fails, retry with the errors fed back. Dedupe items that were mentioned more than once.

7. 7

   Render and export

   Present the summary, the decisions, and a table of action items, and offer markdown and JSON downloads. Make null owners and dates visibly blank so a human knows what to fill in rather than trusting a guess.

8. 8

   Test on a real recording

   Run a genuine 30-plus-minute meeting through the whole pipeline. Check that the summary matches what happened, that action items have the right owners where they were stated, and that nothing was invented. Fix the pipeline until a real meeting comes out clean.

Done when

• ✓A 30-minute recording produces a summary that someone who attended agrees is accurate.
• ✓Action items come back as valid JSON with owner and due date, or null where those were not stated.
• ✓No action item is attributed to a person the meeting never named.
• ✓Long audio that exceeds the context window is handled without dropping the second half.
• ✓The live URL turns a fresh upload into summary plus action items end to end.

Ship it

Dockerize the FastAPI service with Whisper and ffmpeg baked in, and deploy to Hugging Face Spaces or AWS free tier; run heavier transcription on free Colab or Kaggle GPU if needed. Ship a live URL and a README showing a real transcript excerpt next to the generated summary and action-item table. A short demo clip of upload to result makes it tangible.

What it proves: You can build a real multi-stage AI pipeline, chaining speech-to-text with structured LLM summarization, handling long inputs, and coercing messy discussion into validated action items an actual team could use.

Paste this into Cursor, Claude, or ChatGPT and build it step by step.

You are my senior AI engineer pair. Help me build "Meeting summarizer with action items" step by step. The pipeline is speech-to-text then summarization: Whisper (an open-source transcription model) turns audio into text, then an LLM compresses it and extracts action items as structured JSON. Stack: FastAPI, Whisper for transcription (free CPU or Colab/Kaggle GPU), ffmpeg for audio prep, Groq (Llama 3.3 70B) for summary and extraction, Pydantic for the action-item schema, Docker.

Requirements:
1. Accept an audio/video upload and normalize it to 16kHz mono WAV with ffmpeg.
2. Transcribe with Whisper keeping segment timestamps.
3. Chunk long transcripts on natural boundaries so they fit the context window.
4. Summarize faithfully (map over chunks, then reduce) with no invented outcomes.
5. Extract decisions and action items as JSON shaped { task, owner, dueDate }, null when unstated, never inventing an owner; validate and export markdown + JSON.

Work in this order: upload and ffmpeg prep, then Whisper transcription, then chunking, then faithful summarization, then structured action-item extraction, then validation and merge, then render/export. Give me the commands and code for each step and STOP after each so I can test. Do not write the whole app at once.