# Full Build & Test Pipeline

**File:** `.github/workflows/full_pipeline.yml`  
**Purpose:** Comprehensive testing and validation before production  
**Duration:** ~3-8 minutes (cached vs. fresh build)  

## Overview
The full pipeline performs complete testing with all dependencies, including heavy AI/ML libraries. **Runs only on PRs to main** and manual triggers, optimizing resource usage while ensuring comprehensive validation before production integration.

## Technical Details

### Trigger Conditions
```yaml
on:
  pull_request:
    branches: [ "main" ]
  workflow_dispatch:  # Manual trigger only
```

**Strategy:** Comprehensive testing only when integrating to main branch via PR, plus manual trigger for urgent scenarios. No automatic push triggers to optimize resource usage.

### Job Configuration
```yaml
jobs:
  full-test:
    runs-on: ubuntu-latest
    name: Complete Testing Suite
    # Note: Code quality already validated by separate pipeline
```

**Pipeline Philosophy:** Focus on comprehensive testing and integration validation. Code quality is handled by the dedicated Code Quality pipeline that runs on every push.

## Dependency Management

### Core Dependencies
```yaml
# FastAPI ecosystem
fastapi>=0.104.1
uvicorn[standard]>=0.24.0
gunicorn>=21.2.0

# Database
aiomysql>=0.2.0
tortoise-orm[aiomysql]>=0.20.0

# AI/ML Processing
dlib>=19.24.0              # Face detection (heavy build)
opencv-python>=4.8.1.78    # Computer vision
faster-whisper>=1.0.0      # Speech recognition
numpy>=1.24.3              # Numerical computing

# AWS Services
aioboto3>=12.0.0
botocore>=1.34.0

# Monitoring
sentry-sdk[fastapi]>=1.40.0
prometheus-client>=0.19.0
```

### Heavy Dependency Optimization

#### dlib Caching Strategy
```yaml
- name: Cache dlib wheel
  uses: actions/cache@v3
  with:
    path: ~/.cache/pip/wheels
    key: ${{ runner.os }}-dlib-wheel-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-dlib-wheel-
```

**Performance Impact:**
- **First build:** ~8 minutes (compile from source)
- **Cached build:** ~3 minutes (pre-built wheel)
- **Cache hit rate:** ~80%

#### Installation Strategy
```bash
# Force consistent cache location
export PIP_CACHE_DIR=~/.cache/pip

# Try cached wheel first, fallback to source
pip install --find-links ~/.cache/pip/wheels dlib || pip install dlib

# Install remaining dependencies
pip install --prefer-binary -r requirements.txt
```

## Testing Framework

### Test Execution
```yaml
pytest tests/ \
  --cov=app \
  --cov-report=xml \
  --cov-report=term-missing \
  --junitxml=junit.xml \
  -v
```

### Coverage Configuration
```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = [
    "--cov=app",
    "--cov-report=term-missing",
    "--cov-report=html",
    "--cov-report=xml",
]
```

### Test Categories
- **Unit Tests:** Individual component testing
- **Integration Tests:** Service interaction testing
- **API Tests:** Endpoint validation
- **Coverage Analysis:** Code coverage measurement

## Reporting & Analytics

### CodeCov Integration

#### Coverage Upload
```bash
curl -Os https://cli.codecov.io/latest/linux/codecov
chmod +x codecov
./codecov --verbose upload-process \
  --fail-on-error \
  -t ${{ secrets.CODECOV_TOKEN }} \
  -n 'ai-feedback'-${{ github.run_id }} \
  -F ai-feedback \
  --file coverage.xml
```

#### Test Results Analytics
```yaml
- name: Upload test results to Codecov
  uses: codecov/test-results-action@v1
  with:
    token: ${{ secrets.CODECOV_TOKEN }}
    file: junit.xml
    flags: ai-feedback
    name: ai-feedback-test-results
```

### GitHub Summary Report
```markdown
## 🚀 Full Pipeline Results

✅ **Full Test Suite**: PASSED
✅ **Ready for Production**: YES

Coverage: 27%
Tests Passed: 54/54
Duration: 3m 42s
```

## Performance Characteristics

### Execution Timeline
1. **Setup & Checkout:** ~30 seconds
2. **Python & Cache Setup:** ~45 seconds
3. **Dependency Installation:** 
   - Fresh: ~8 minutes (dlib compilation)
   - Cached: ~2 minutes (wheel reuse)
4. **Test Execution:** ~60 seconds
5. **Reporting:** ~30 seconds

**Total Duration:** ~3-10 minutes (testing only - quality pre-validated)

### Resource Requirements
- **CPU:** High during dlib compilation
- **Memory:** ~4GB peak (AI/ML libraries)
- **Storage:** ~2GB cache (wheels + dependencies)
- **Network:** ~500MB downloads (fresh build)

## Exit Criteria

### Success Conditions
✅ All tests pass  
✅ Coverage reports generated  
✅ No critical failures  
✅ CodeCov upload successful  

### Failure Conditions
❌ Test failures  
❌ Import errors  
❌ Coverage below threshold  
❌ Build timeouts  

## Security & Compliance

### Token Management
```yaml
secrets:
  CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
```
- **Scope:** Repository access only
- **Rotation:** Annual review
- **Access:** Repository maintainers only

### Dependency Security
- **Vulnerability scanning:** Integrated with dependency installation
- **Version pinning:** Explicit version constraints
- **License compliance:** MIT-compatible dependencies only

## Local Development

### Full Test Suite
```bash
# Install all dependencies
pip install -r requirements.txt
pip install pytest pytest-cov pytest-asyncio httpx

# Run complete test suite
pytest tests/ --cov=app --cov-report=term-missing -v

# Generate coverage report
pytest tests/ --cov=app --cov-report=html
open htmlcov/index.html
```

### Dependency Management
```bash
# Check for dependency issues
pip check

# Update requirements
pip-compile requirements.in

# Security audit
pip audit
```

## Monitoring & Alerting

### Performance Metrics
- **Duration Target:** <5 minutes (95th percentile)
- **Success Rate:** >98%
- **Cache Hit Rate:** >80%
- **Coverage Trend:** Maintain or improve

### Alerting Thresholds
- **Pipeline Failure:** Immediate notification
- **Duration >10 minutes:** Investigation required
- **Cache Miss Rate >30%:** Optimization needed
- **Coverage Drop >5%:** Review required

## Optimization Strategies

### Current Optimizations
1. **Aggressive caching:** dlib wheel storage
2. **Pip cache:** Package download optimization
3. **Parallel execution:** Independent test modules
4. **Binary preferences:** Pre-compiled packages

### Future Improvements
1. **Multi-stage caching:** Separate AI/ML dependencies
2. **Container optimization:** Custom base images
3. **Test parallelization:** pytest-xdist integration
4. **Incremental testing:** Only test changed modules

## Troubleshooting

### Common Issues

#### dlib Compilation Timeout
**Symptoms:**
- Pipeline timeout after 10+ minutes
- "Building wheel for dlib" stuck

**Solutions:**
1. Check cache configuration
2. Verify wheel storage path
3. Clear cache and rebuild
4. Use pre-compiled wheels if available

#### Memory Issues
**Symptoms:**
- Out of memory errors
- Process killed during tests

**Solutions:**
1. Reduce test parallelization
2. Skip memory-intensive tests in CI
3. Use swap space configuration
4. Optimize test data size

#### Import Errors
**Symptoms:**
- "ModuleNotFoundError" during tests
- Missing dependencies

**Solutions:**
1. Verify requirements.txt completeness
2. Check dependency installation logs
3. Validate virtual environment setup
4. Test locally with fresh environment

### Debug Commands
```bash
# Check installed packages
pip list

# Verify test discovery
pytest --collect-only

# Run with verbose output
pytest -v -s

# Check coverage details
pytest --cov=app --cov-report=term-missing --cov-fail-under=25
```

## Best Practices

### ✅ Reliability
- Comprehensive dependency installation
- Robust error handling
- Graceful failure modes
- Detailed logging and reporting

### ✅ Performance
- Smart caching strategies
- Binary package preferences
- Parallel execution where possible
- Resource optimization

### ✅ Maintainability
- Clear documentation
- Version pinning
- Reproducible builds
- Automated reporting