Problem
Most digital content isn’t truly accessible for deaf users beyond captions. Captions alone can miss context, tone, and sign-language-specific communication needs.
Solution
I built Sign-ify as a multi-modal accessibility platform that accepts YouTube links, uploaded audio, or text and produces sign-language-oriented output with AI-assisted guidance.
My Role
Lead Full-Stack AI Engineer.
- Architected the FastAPI backend for low-latency media processing
- Integrated STT/TTS and LLM workflows for interpretation support
- Built an analytics dashboard for usage and quality monitoring
- Designed accessible, low-friction interaction flows
Technical Decisions
- FastAPI for performant async handling of media-heavy requests
- Pipeline architecture (STT → NLP → output) for modular quality tuning
- Operational analytics to track failure points and quality drift
- Model abstraction layer to swap providers without product rewrites
Impact
- Brought input-to-output translation down to near real time
- Increased accessibility sessions after adding multi-input support
- Improved output quality through analytics-led iteration on edge cases
Key Features
- Multi-modal translation — Converts YouTube URLs, uploaded audio, and raw text into sign language.
- OpenAI chatbot integration — An assistant that explains sign language grammar and cultural nuance.
- Analytics dashboard — An admin interface to monitor system performance and usage metrics.
- Speech-to-sign — Real-time processing using Speech-to-Text (STT) and Text-to-Speech (TTS) pipelines.
- FastAPI backend — Handles high concurrency and data flow between the AI models and the frontend.
Tech Stack
- Frontend: React.js, Tailwind CSS
- Backend: FastAPI (Python), Node.js
- AI Services: OpenAI API (GPT-4), Whisper (STT), Google/Azure TTS
- Data: YouTube Data API, custom sign language libraries