Skip to main content

​
Overview

FKApi is a Django REST API built to provide structured access to football kit data. The system follows a layered architecture pattern with clear separation of concerns across presentation, business logic, and data access layers.

​
Architecture Diagram

┌─────────────────────────────────────────────────────────────┐
│                        Client Layer                          │
│  (Web Browsers, Mobile Apps, API Clients)                   │
└────────────────────────┬────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│                      Django Application                      │
│  ┌──────────────────────────────────────────────────────┐  │
│  │              URL Routing (urls.py)                    │  │
│  └──────────────────┬───────────────────────────────────┘  │
│                     │                                        │
│  ┌──────────────────▼───────────────────────────────────┐  │
│  │              Middleware Layer                          │  │
│  │  • Rate Limiting                                       │  │
│  │  • Performance Monitoring                             │  │
│  └──────────────────┬───────────────────────────────────┘  │
│                     │                                        │
│  ┌──────────────────▼───────────────────────────────────┐  │
│  │              API Layer (Django Ninja)                  │  │
│  │  • REST API Endpoints                                 │  │
│  │  • Request Validation                                │  │
│  │  • Response Serialization                            │  │
│  └──────────────────┬───────────────────────────────────┘  │
│                     │                                        │
│  ┌──────────────────▼───────────────────────────────────┐  │
│  │              Service Layer                             │  │
│  │  • Business Logic                                     │  │
│  │  • Data Processing                                    │  │
│  │  • Scraping Service                                   │  │
│  └──────────────────┬───────────────────────────────────┘  │
│                     │                                        │
│  ┌──────────────────▼───────────────────────────────────┐  │
│  │              Data Access Layer                         │  │
│  │  • Django ORM                                         │  │
│  │  • Model Queries                                      │  │
│  └──────────────────┬───────────────────────────────────┘  │
└─────────────────────┼───────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────┐
│                    Data Storage Layer                        │
│  ┌──────────────┐  ┌──────────────┐                       │
│  │  PostgreSQL  │  │    Redis     │                       │
│  │   Database   │  │    Cache     │                       │
│  └──────────────┘  └──────────────┘                       │
└─────────────────────────────────────────────────────────────┘

​
Core Components

​
1. Client Layer

The entry point for all external interactions:
  • Web browsers accessing the API
  • Mobile applications
  • Third-party API clients
  • API testing tools (Postman, cURL)

​
2. URL Routing

Django’s URL dispatcher maps incoming requests to appropriate endpoints:
  • Defined in fkapi/urls.py
  • Routes API requests to Django Ninja endpoints
  • Handles admin interface routing

​
3. Middleware Layer

Two custom middleware components provide cross-cutting functionality:
Prevents API abuse by limiting requests per IP address:
  • Tracks request counts per IP
  • Configurable rate limits
  • Whitelist support via API_RATE_LIMIT_WHITELIST
  • Returns 429 status code when limits exceeded
Monitors application performance:
  • Tracks response times
  • Logs slow queries (threshold: SLOW_QUERY_THRESHOLD)
  • Identifies slow responses (threshold: SLOW_RESPONSE_THRESHOLD)
  • Optional database query logging via LOG_DB_QUERIES

​
4. API Layer (Django Ninja)

Django Ninja provides a modern, FastAPI-style API framework:
  • RESTful endpoint definitions
  • Automatic request validation using Pydantic
  • Response serialization
  • OpenAPI/Swagger documentation generation
  • Type-safe API development

​
5. Service Layer

Business logic lives in service classes under core/services/:
  • ScrapingService: Orchestrates web scraping operations
  • KitsService: Handles kit-related business logic
  • ClubsService: Manages club operations
This separation enables:
  • Testable business logic
  • Reusable components across endpoints
  • Clear separation of concerns

​
6. Data Access Layer

Django ORM provides database abstraction:
  • Model definitions in core/models.py
  • Query optimization with select_related() and prefetch_related()
  • Database indexes on frequently queried fields
  • Transaction management

​
7. Data Storage

PostgreSQL Database

Primary data store for all structured data:
  • Clubs, Kits, Seasons
  • Brands, Competitions
  • Colors and Variations
  • Connection pooling enabled (CONN_MAX_AGE: 60)
  • Keepalive settings for connection stability

Redis Cache

Optional caching layer:
  • Reduces database load
  • Stores API responses
  • Cache timeout configurations (5m to 24h)
  • Automatically enabled when Celery is active
  • Falls back to local memory cache if unavailable
Images (kit photos, logos) are stored as URLs pointing to footballkitarchive.com, not served locally.

​
Data Flow

​
API Request Flow

1

Client Request

Client sends HTTP request to API endpoint
2

URL Routing

Django’s URL dispatcher routes request to appropriate endpoint
3

Middleware Processing

Request passes through rate limiting and performance monitoring
4

API Endpoint

Django Ninja validates request and calls appropriate function
5

Service Layer

Business logic executes (validation, transformation)
6

Data Access

Django ORM queries database (checks cache first if enabled)
7

Database Query

PostgreSQL returns requested data
8

Response Serialization

Data serialized to JSON format
9

Middleware Response

Performance metrics logged, headers added
10

Client Response

JSON response returned to client

​
Scraping Data Flow

The scraping process ingests data from footballkitarchive.com:
  1. Trigger: Management command or Celery task initiates scraping
  2. HTTP Request: http_get() fetches page with retry logic and proxy support
  3. HTML Parsing: BeautifulSoup4 extracts structured data from HTML
  4. Data Processing: Service layer validates and transforms data
  5. Database Save: ORM creates or updates records within transactions
  6. Cache Invalidation: Django signals clear affected cache entries

​
Design Patterns

​
Service Layer Pattern

Separates business logic from presentation layer:
  • Located in core/services/
  • Promotes code reusability
  • Enhances testability
  • Example: ScrapingService.process_kit_data()

​
Repository Pattern

Implemented via Django ORM:
  • Data access abstraction
  • Custom model managers for complex queries
  • Query optimization methods

​
Middleware Pattern

Handles cross-cutting concerns:
  • Rate limiting applies to all requests
  • Performance monitoring on every response
  • Executes before and after view processing

​
Signal Pattern

Event-driven cache invalidation:
  • Django signals trigger on model changes
  • Automatic cache clearing on save/delete
  • Maintains cache consistency

​
Factory Pattern

Configurable object creation:
  • HTTP session creation with proxy support
  • Scraper configuration based on environment

​
Technology Stack

Django 5.0

Web framework

Django Ninja

REST API framework

PostgreSQL

Primary database

Redis

Cache layer

Celery

Task queue (optional)

BeautifulSoup4

HTML parsing

​
Configuration

Key environment variables from settings.py:
VariablePurposeDefault
DJANGO_DEBUGDebug modeTrue
DJANGO_SECRET_KEYSecurity keyAuto-generated in dev
DJANGO_ALLOWED_HOSTSAllowed hostslocalhost,127.0.0.1
ENABLE_CELERYEnable task queueBased on availability
USE_REDIS_CACHEUse Redis for cachingFalse
SLOW_QUERY_THRESHOLDSlow query threshold (seconds)0.5
SLOW_RESPONSE_THRESHOLDSlow response threshold (seconds)1.0
API_RATE_LIMIT_WHITELISTWhitelisted IPsNone

​
Performance Optimizations

Multi-tier caching with configurable timeouts:
  • CACHE_TIMEOUT_SHORT: 5 minutes
  • CACHE_TIMEOUT_MEDIUM: 30 minutes
  • CACHE_TIMEOUT_LONG: 1 hour
  • CACHE_TIMEOUT_VERY_LONG: 24 hours
  • Connection pooling (CONN_MAX_AGE: 60)
  • Keepalive settings for connection stability
  • Indexes on frequently queried fields
  • select_related() and prefetch_related() for joins
  • Database indexes on:
    • Club: name, country, slug
    • Kit: name, team+season, main_img_url, rating
    • Type_K: category_order, is_goalkeeper, order_priority
    • Competition: name, country

​
Security Features

  • Rate Limiting: Prevents API abuse
  • Optional API Key Authentication: Via ninja-apikey
  • SQL Injection Protection: Django ORM parameterized queries
  • XSS Protection: Django template escaping
  • CSRF Protection: Enabled for admin, disabled for API endpoints
  • Secure Headers: In production mode:
    • SECURE_SSL_REDIRECT
    • SESSION_COOKIE_SECURE
    • CSRF_COOKIE_SECURE
    • SECURE_HSTS_SECONDS: 1 year
    • X_FRAME_OPTIONS: DENY

​
Deployment Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Load Balancer                           │
└────────────────────────┬────────────────────────────────────┘
                         │
         ┌───────────────┴───────────────┐
         │                               │
         ▼                               ▼
┌─────────────────┐           ┌─────────────────┐
│  Django App 1   │           │  Django App 2   │
│  (Gunicorn)     │           │  (Gunicorn)     │
└────────┬────────┘           └────────┬────────┘
         │                               │
         └───────────────┬───────────────┘
                         │
         ┌───────────────┴───────────────┐
         │                               │
         ▼                               ▼
┌─────────────────┐           ┌─────────────────┐
│   PostgreSQL    │           │     Redis      │
│   (Primary)     │           │     Cache      │
└─────────────────┘           └─────────────────┘
Multiple Django application instances can run behind a load balancer. Redis ensures cache consistency across instances.

Data Models

Learn about database models and relationships

Scraping Architecture

Understand web scraping implementation