Storybook Library API

Description

Cortical Visual Impairment (CVI) is a visual processing disability where brain can process realistic photos much easier than illustrations, and is the leading cause of visual impairments in the US. Children with CVI have an inability to process abstract shapes like drawings and cartoons into meaningful object associations, making it difficult to learn how to read and limiting their ability to thrive. Storybook Library solves this by generating photorealistic images from literature, with limited characters on a pure black background.

This is achieved as a multi-step process that involves: parsing the text with an LLM to identify characters, clothing, scenes and themes, rendering these photo-realistically using an AI image model, monitoring and adjusting prompts for safety, and maintaining character consistency and continuity from scene to scene using a database record. Designed, architected, and developed from concept to delivery for educators working across 12 school districts in central Connecticut.

Visual Demo

Key Technical Decisions

LLM text preprocessing before illustration

Raw story text is preprocessed through GPT-6 Luna before image generation. The LLM extracts scene boundaries, identifies characters present, and generates detailed visual descriptions — converting narrative prose into structured illustration prompts. This separation ensures the image model receives consistent, visually-oriented instructions rather than raw literary text.

Character consistency through context propagation

A significant challenge with AI illustration is that each render is independent, so a character can look completely different from one page to the next. The pipeline solves this by maintaining a persistent character registry per book. Character descriptions are stored in the database and evolve — when a character's appearance is approved after regeneration, the reference description updates to match. Every image prompt includes the full character context for all characters in that scene, plus reference images from previously approved renders.

Measured model evaluation over spec-sheet pricing

When OpenAI scheduled its image models for retirement, the same 34-page book was generated on three candidates (GPT image-2.5 Flare, Sunburst, and Gemini Nano Banana 2.1) and compared on cost, character consistency, and safety-filter behavior. Token usage was measured from live API calls rather than estimated: the nominally 'cheaper per token' model used far fewer tokens per image, and reference images, not output, turned out to dominate cost. Sunburst won on character consistency at the same price as Flare.

Provider-agnostic safety recovery

OpenAI rejects unsafe prompts with an HTTP 400, while Gemini returns a successful response with no image and a PROHIBITED_CONTENT finish reason. The worker normalizes both into one safety signal, so every provider shares the same recovery chain: an LLM rewrites the prompt, then reimagines the scene from a different angle, before a page is marked failed.

Deprecation-proof model remapping

Retired models are swapped at generation time, within the same provider, rather than by rewriting stored data. Each book keeps a record of the model that produced its existing pages, while new pages and regenerations use the current model. Regeneration also uses the book's own model and quality, so replacement pages match the lighting and style of the rest of the book.

Separate API and Worker services

Cost control pattern — the deployed API is intentionally read-only. The AI pipeline runs locally where costs can be monitored. The public endpoint only serves pre-processed content.

Upstash Redis polling with sorted-set delayed jobs

Upstash has no blocking pop (BLPOP). The worker polls with a 5-second interval and uses a sorted set with timestamp scores for rate limit backoff, promoting delayed jobs when their wait time expires.

Hand-rolled image regeneration UX

Admin can click any thumbnail to open a lightbox, edit the AI prompt, regenerate, then approve or reject in a side-by-side comparison view. Rejected images are deleted from R2; approved ones replace the original.

By the Numbers

15+ fairy tales catalogued across 2 public domain collections

4 classic novels with chapter-by-chapter processing plans

3-attempt safety fallback (rewrite → reimagine) across OpenAI and Gemini

Head-to-head evaluation of 3 image models on a 34-page book with measured per-image cost

Full E2E test suite with Playwright (29 specs, 154 tests)