## Documentation Index

This directory contains everything you need to understand, run, and operate the AI Feedback FastAPI service.

### Table of Contents
- Architecture and Flow: `architecture.md`
- Services Overview: `services.md`
- Queue Processing: `queue.md`
- S3 and File Handling: `s3-file-handling.md`
- Resources (Models & Word Lists): `resources.md`
- Quickstart (Local & Docker): `quickstart.md`
- Migration Notes (Legacy → FastAPI): `migration.md` (optional, if present)

---

## Overview (What this service does)

The service ingests recorded interview media, analyzes it, and produces legacy-compatible JSON with:
- Speech metrics (words, pace, pauses, volume)
- Content metrics (filler/power/negative/um words + timestamps/phrases)
- Visual metrics (gaze left/center/right, smiling %, lighting)

It replaces the legacy Flask service while preserving output shape and semantics expected by downstream systems.

Key properties:
- Legacy pupil-based gaze tracking with ROI-centered eyes and smoothing
- Whisper transcription with timestamps and explicit start/end logs
- Background queue poller (default 30s) + timeout monitor
- Processes only jobs considered ready by upstream: `status='0'` and `data_process='1'`

---

## Architecture (Start → End)

See `architecture.md` for the full Mermaid diagram. High-level steps:
1. Upstream enqueues a job in MySQL (`status='0'`, `data_process='1'` when upstream marks ready)
2. Background poller (30s) or `POST /api/v1/queue/process` triggers a pass
3. `QueueProcessor` fetches ready jobs and claims them (`status='1'`)
4. `VideoProcessor`/`AudioProcessor` download media via `FileService` (S3)
5. `VideoAnalysisService` orchestrates:
   - `AnalysisUtils` (Whisper transcription + content)
   - `GazeTrackingService` (legacy pupil logic)
   - `FaceAnalysisService` (smile, light)
6. Results are persisted (`FeedbackRepository`) and notifications sent
7. Jobs are marked completed (`status='3'`) or failed (`status='2'`)

---

## Services (What lives where)

See `services.md` for details. Highlights:
- `app/services/gaze_tracking_service.py`: Legacy pupil-based gaze, ROI-centered eyes, smoothing, quality gating
- `app/services/face_analysis_service.py`: Haar cascades (smile), HSV light analysis, legacy thresholds
- `app/services/analysis_utils.py`: Whisper timestamps, silence detection (3200ms/-40dB), content metrics; Whisper start/end logs
- `app/services/video_analysis.py`: Orchestrates audio, transcription, gaze, face, lighting; emits legacy-compatible JSON
- `app/services/video_processor.py` / `audio_processor.py`: S3 download, run analysis, persist results
- `app/services/background_tasks.py`: Timeout monitor + queue poller

---

## Queue Processing (How jobs move)

See `queue.md` for the full write-up. Essentials:
- Pending selector: `status='0'` AND `data_process='1'`
- Polling: every 30 seconds by default (configurable in `BackgroundTaskService`)
- Endpoints: `/api/v1/queue/process`, `/api/v1/queue/timeout-check`, `/api/v1/queue/status`
- Recommended DB index: `(status, data_process)`

Glossary:
- `status='0'`: pending; `status='1'`: processing; `status='2'`: failed; `status='3'`: completed
- `data_process='1'`: upstream has confirmed media is ready/available (S3)

---

## Resources (Models & Word Lists)

See `resources.md` for required files under `app/resources`:
- Models: dlib shape predictor, OpenCV cascades (face/smile)
- Word Lists: filler, filler_phrase, power, power_phrase, negative, um_words

---

## Quickstart (Developer notes)

See `quickstart.md` for full instructions. Summary:
1. `make install-dev` or `uv sync`
2. Ensure `app/resources` contains required models/word lists
3. Configure env via `config/env.local` (or Docker compose)
4. `make run-dev` and open `http://localhost:9001/docs`

---

## Troubleshooting

- dlib / OpenCV model load issues: verify model paths under `app/resources/models`
- Whisper performance: logs show when transcription starts/ends and elapsed time
- S3/file errors: all downloads go through `FileService`; check logs for bucket/key and HTTP statuses
- Queue looks idle: verify records are `status='0'` and `data_process='1'`; poller interval defaults to 30s

---

## Recent Changes (v2.x)

- Gaze tracking aligned to legacy (pupil-based ratio, ROI center, thresholds 0.35/0.65) with smoothing
- Whisper transcription logs (start/end, language, segments/words, elapsed)
- Background queue poller added; endpoints documented