r/django • • 14h ago

Built a prompt engineering workspace with Django 5 & DRF — how are you handling external LLM latency and streaming?

0 Upvotes

Hey everyone,

Over the past few weeks, I have been building an open prompt workspace and chaining platform called EvokeBase using Django 5 and Django REST Framework (DRF) on the backend, paired with a React 18 + Vite frontend.

Most LLM wrappers in tutorials use FastAPI or Node, but I wanted to explore how far we can push Django 5's modern async capabilities and DRF viewsets for production-ready AI workflows.

Backend Architecture & Stack:

- Framework: Django 5.x + Django REST Framework (DRF)

- Authentication: JWT via djangorestframework-simplejwt (with refresh token rotation and blacklisting)

- Database: SQLite (local dev) / PostgreSQL (production)

- Frontend: React 18, Tailwind CSS, Framer Motion

- Hosting: Vercel (Frontend) + Cloud backend

Key Django Design Patterns Used:

  1. Prompt Template & Variable Schema:

Instead of storing raw text and parsing on every request, we store template metadata and extract {{variable}} slots client-side with debounced regex, passing structured payloads to DRF serializers.

2. Context Injection Pipeline:

Built custom service layers where project-specific tone parameters (target audience, formatting constraints, banned keywords) are automatically merged into the system prompt payload before dispatching to model endpoints.

3. Multi-Model Chaining Logic:

Orchestrated an asynchronous pipeline where Node 1 output (e.g. Gemini extracting structured JSON) feeds directly as typed context into Node 2 (e.g. Claude drafting long-form copy).

Questions for the Django Community:

As the platform scales, I would love to learn from your production experiences:

  1. Handling LLM Latency in DRF: When waiting 3 to 8 seconds for external model APIs (Anthropic/OpenAI/Gemini), do you prefer standard DRF views with Celery + Redis workers, or have you migrated to Django 5 native ASGI async views (StreamingHttpResponse)?
  2. Token Security: For users who bring their own API keys, what is your preferred pattern in Django — field-level encryption with django-cryptography or vault solutions?
  3. Database Versioning: For prompt iteration history, do you prefer django-simple-history or dedicated versioning tables with parent-child foreign keys?

If you would like to test the live UI and responsiveness, you can check it out here:

Live Demo: https://www.evokebase.com

Would love any feedback on the architecture, API design, or Django-specific optimizations!