MathVizAI: Turning mathematical questions into narrated animations

01 / Problem & ownership

Connecting multi-stage reasoning, narration, and programmatic animation generation.

02 / How it works

From mathematical question to final rendered video.

1. Mathematical Solver & Review: A solver generates a mathematical explanation, and a separate evaluator reviews it. The pipeline can retry using the evaluator’s feedback.

2. Script & Narration: The script writer converts the explanation into narration segments. VibeVoice generates the audio, after which the system estimates phrase timings using the audio duration and script.

3. Scene Generation Informed by Audio Timing: The visual developer uses those timings, together with retrieved Manim examples, to generate scene code.

4. Rendering & Synchronization: The pipeline renders the individual scenes and synchronises the audio and video for final FFmpeg assembly.

(Note on architectural ordering: Generating audio before the Manim script allows actual duration information to guide visual scene pacing).

03 / Engineering decisions

Retrieval-assisted generation, scene-level checks, and narration-informed timing.

Retrieval-assisted generation: The visual developer can retrieve relevant code examples rather than relying only on the model’s unaided knowledge of animation APIs.

Scene-level checks and retries: Generated scenes can undergo review and a runtime verification step. Failed verification supplies feedback for another attempt, and unsuccessful scene generation can stop the remaining scenes.

Timing informed by narration: The system uses generated audio duration to estimate phrase timing and inform animation durations. This coordinates the media pipeline without claiming word-perfect alignment.

# Multi-Stage Pipeline Loop with Review & Fallback
def execute_pipeline(problem_statement: str) -> str:
    # Step 1: Solve with review-and-retry loop
    solution = None
    for attempt in range(MAX_RETRIES):
        candidate = solver.solve(problem_statement)
        evaluation = evaluator.review(candidate)
        if evaluation.is_valid:
            solution = candidate
            break
    if not solution:
        solution = candidate  # Fallback to last candidate on retry exhaustion
        
    # Step 2: Script and audio duration estimation
    segments = script_writer.segment_and_write(solution)
    audio_segments = [tts.generate(seg.narration) for seg in segments]
    
    # Step 3 & 4: Scene generation informed by narration timing & assembly
    rendered_scenes = []
    for seg, audio in zip(segments, audio_segments):
        scene_code = rag_generator.generate_scene(seg, duration=audio.duration)
        video = renderer.render(scene_code)
        rendered_scenes.append((video, audio))
        
    return assembly.concatenate(rendered_scenes)

04 / Evidence & limitations

Demonstrated artifacts and operational boundaries.

The repository includes sample videos demonstrating Taylor series and gradient descent. These illustrate the type of output the pipeline can produce, rather than a measured success rate across arbitrary mathematical questions.

LLM review is not formal mathematical verification. The current solver loop can continue with its final attempt after reaching the retry limit, even when the evaluator reports issues. Generated explanations and visualisations therefore still require review before educational use.

Phrase timing is estimated proportionally from word counts rather than established through word-level audio alignment. Runtime checks help identify execution failures, but a successful render does not by itself establish mathematical or visual correctness.