Showing posts with label Golang. Show all posts
Showing posts with label Golang. Show all posts

Wednesday, 15 October 2025

Building Custom Kubernetes Operators in Go - Complete 2025 Guide

October 15, 2025 0

Writing a Custom Kubernetes Operator in Go for Complex Application Orchestration

Kubernetes Custom Operator in Go - Architecture diagram showing Go programming language orchestrating multiple Kubernetes pods and containers with custom resource definitions for automated application management

Kubernetes operators have revolutionized how we manage complex stateful applications in cloud-native environments. As we move into 2025, the ability to build custom operators has become an essential skill for platform engineers and DevOps professionals. In this comprehensive guide, we'll dive deep into creating a production-ready Kubernetes operator using Go and the Operator SDK, complete with advanced patterns for handling complex application orchestration, automatic recovery, and intelligent scaling.

🚀 Why Custom Kubernetes Operators Matter in 2025

Kubernetes operators represent the pinnacle of cloud-native automation. They encode human operational knowledge into software that can manage complex applications autonomously. With the rise of AI workloads and microservices architectures, custom operators have become crucial for:

  • AI/ML Pipeline Management: Orchestrating complex training and inference workflows
  • Database Operations: Automated backups, scaling, and failover for stateful data systems
  • Multi-cluster Deployments: Managing applications across hybrid cloud environments
  • Cost Optimization: Intelligent scaling based on custom metrics and business logic
  • GitOps Integration: Seamless integration with modern deployment workflows

The latest Operator Framework enhancements in 2025 have made building operators more accessible than ever, while maintaining the power and flexibility needed for enterprise-grade applications.

🔧 Setting Up Your Operator Development Environment

Before we dive into code, let's set up our development environment with the latest tools available in 2025:

  • Kubernetes 1.30+: Latest features and API stability
  • Operator SDK 2.8+: Enhanced scaffolding and testing capabilities
  • Go 1.22+: Improved generics and performance optimizations
  • Kubebuilder 4.0+: Streamlined CRD generation
  • Kind 0.22+: Local Kubernetes cluster for testing

💻 Code Example: Basic Operator Structure


package main

import (
    "context"
    "fmt"
    "os"
    "time"

    appsv1 "k8s.io/api/apps/v1"
    corev1 "k8s.io/api/core/v1"
    "k8s.io/apimachinery/pkg/api/errors"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/runtime"
    "k8s.io/apimachinery/pkg/types"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/client"
    "sigs.k8s.io/controller-runtime/pkg/log"
    "sigs.k8s.io/controller-runtime/pkg/log/zap"

    mygroupv1 "github.com/your-org/app-operator/api/v1"
)

// AppOperatorReconciler reconciles a AppOperator object
type AppOperatorReconciler struct {
    client.Client
    Scheme *runtime.Scheme
}

//+kubebuilder:rbac:groups=mygroup.example.com,resources=appoperators,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=mygroup.example.com,resources=appoperators/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=mygroup.example.com,resources=appoperators/finalizers,verbs=update
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete

func (r *AppOperatorReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    logger := log.FromContext(ctx)
    
    // Fetch the AppOperator instance
    var appOp mygroupv1.AppOperator
    if err := r.Get(ctx, req.NamespacedName, &appOp); err != nil {
        if errors.IsNotFound(err) {
            logger.Info("AppOperator resource not found. Ignoring since object must be deleted")
            return ctrl.Result{}, nil
        }
        logger.Error(err, "Failed to get AppOperator")
        return ctrl.Result{}, err
    }

    // Check if deployment already exists, if not create a new one
    found := &appsv1.Deployment{}
    err := r.Get(ctx, types.NamespacedName{Name: appOp.Name, Namespace: appOp.Namespace}, found)
    if err != nil && errors.IsNotFound(err) {
        // Define a new deployment
        dep := r.deploymentForAppOperator(&appOp)
        logger.Info("Creating a new Deployment", "Deployment.Namespace", dep.Namespace, "Deployment.Name", dep.Name)
        err = r.Create(ctx, dep)
        if err != nil {
            logger.Error(err, "Failed to create new Deployment", "Deployment.Namespace", dep.Namespace, "Deployment.Name", dep.Name)
            return ctrl.Result{}, err
        }
        // Deployment created successfully - return and requeue
        return ctrl.Result{Requeue: true}, nil
    } else if err != nil {
        logger.Error(err, "Failed to get Deployment")
        return ctrl.Result{}, err
    }

    // Ensure deployment replicas match the spec
    size := appOp.Spec.Replicas
    if *found.Spec.Replicas != size {
        found.Spec.Replicas = &size
        err = r.Update(ctx, found)
        if err != nil {
            logger.Error(err, "Failed to update Deployment", "Deployment.Namespace", found.Namespace, "Deployment.Name", found.Name)
            return ctrl.Result{}, err
        }
    }

    // Update status if needed
    if appOp.Status.AvailableReplicas != found.Status.AvailableReplicas {
        appOp.Status.AvailableReplicas = found.Status.AvailableReplicas
        err := r.Status().Update(ctx, &appOp)
        if err != nil {
            logger.Error(err, "Failed to update AppOperator status")
            return ctrl.Result{}, err
        }
    }

    return ctrl.Result{}, nil
}

// deploymentForAppOperator returns an AppOperator Deployment object
func (r *AppOperatorReconciler) deploymentForAppOperator(a *mygroupv1.AppOperator) *appsv1.Deployment {
    ls := labelsForAppOperator(a.Name)
    replicas := a.Spec.Replicas

    dep := &appsv1.Deployment{
        ObjectMeta: metav1.ObjectMeta{
            Name:      a.Name,
            Namespace: a.Namespace,
        },
        Spec: appsv1.DeploymentSpec{
            Replicas: &replicas,
            Selector: &metav1.LabelSelector{
                MatchLabels: ls,
            },
            Template: corev1.PodTemplateSpec{
                ObjectMeta: metav1.ObjectMeta{
                    Labels: ls,
                },
                Spec: corev1.PodSpec{
                    Containers: []corev1.Container{{
                        Image: a.Spec.Image,
                        Name:  a.Name,
                        Ports: []corev1.ContainerPort{{
                            ContainerPort: a.Spec.Port,
                            Name:          "http",
                        }},
                    }},
                },
            },
        },
    }
    // Set AppOperator instance as the owner and controller
    ctrl.SetControllerReference(a, dep, r.Scheme)
    return dep
}

// labelsForAppOperator returns the labels for selecting the resources
func labelsForAppOperator(name string) map[string]string {
    return map[string]string{"app": "appoperator", "appoperator_cr": name}
}

// SetupWithManager sets up the controller with the Manager.
func (r *AppOperatorReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&mygroupv1.AppOperator{}).
        Owns(&appsv1.Deployment{}).
        Complete(r)
}

func main() {
    opts := zap.Options{
        Development: true,
    }
    opts.BindFlags(flag.CommandLine)
    flag.Parse()

    ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts)))

    var metricsAddr string
    var enableLeaderElection bool
    var probeAddr string
    flag.StringVar(&metricsAddr, "metrics-bind-address", ":8080", "The address the metric endpoint binds to.")
    flag.StringVar(&probeAddr, "health-probe-bind-address", ":8081", "The address the probe endpoint binds to.")
    flag.BoolVar(&enableLeaderElection, "leader-elect", false,
        "Enable leader election for controller manager.")

    mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
        Scheme:                 scheme,
        MetricsBindAddress:     metricsAddr,
        Port:                   9443,
        HealthProbeBindAddress: probeAddr,
        LeaderElection:         enableLeaderElection,
        LeaderElectionID:       "app-operator-lock",
    })
    if err != nil {
        setupLog.Error(err, "unable to start manager")
        os.Exit(1)
    }

    if err = (&AppOperatorReconciler{
        Client: mgr.GetClient(),
        Scheme: mgr.GetScheme(),
    }).SetupWithManager(mgr); err != nil {
        setupLog.Error(err, "unable to create controller", "controller", "AppOperator")
        os.Exit(1)
    }

    if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
        setupLog.Error(err, "problem running manager")
        os.Exit(1)
    }
}

  

🎯 Advanced Operator Patterns for Complex Applications

Modern applications require sophisticated orchestration patterns. Here are some advanced techniques we can implement in our custom operator:

  • StatefulSet Management: Handling stateful applications with persistent storage
  • Cross-resource Coordination: Managing dependencies between different Kubernetes resources
  • Health Checking: Custom health checks beyond standard readiness/liveness probes
  • Rolling Updates with Validation: Safe deployment strategies with pre/post checks
  • External System Integration: Coordinating with cloud services and external APIs

💻 Code Example: Advanced State Management


// Advanced state management with conditions and events
func (r *AppOperatorReconciler) handleApplicationState(ctx context.Context, appOp *mygroupv1.AppOperator) (ctrl.Result, error) {
    logger := log.FromContext(ctx)
    
    // Check current application state
    currentState := r.assessApplicationHealth(ctx, appOp)
    
    // Update conditions based on current state
    switch currentState {
    case ApplicationHealthy:
        r.updateCondition(appOp, mygroupv1.ConditionReady, metav1.ConditionTrue, "ApplicationRunning", "All components are healthy")
        r.Recorder.Event(appOp, corev1.EventTypeNormal, "Healthy", "Application is running smoothly")
        
    case ApplicationDegraded:
        r.updateCondition(appOp, mygroupv1.ConditionReady, metav1.ConditionFalse, "ComponentsUnhealthy", "Some components are degraded")
        return r.handleDegradedState(ctx, appOp)
        
    case ApplicationRecovering:
        r.updateCondition(appOp, mygroupv1.ConditionReady, metav1.ConditionFalse, "RecoveryInProgress", "Application is recovering")
        return r.initiateRecovery(ctx, appOp)
        
    case ApplicationScaling:
        r.updateCondition(appOp, mygroupv1.ConditionReady, metav1.ConditionFalse, "ScalingInProgress", "Application is scaling")
        return r.handleScaling(ctx, appOp)
    }
    
    return ctrl.Result{}, nil
}

// Intelligent scaling based on custom metrics
func (r *AppOperatorReconciler) handleIntelligentScaling(ctx context.Context, appOp *mygroupv1.AppOperator) (ctrl.Result, error) {
    logger := log.FromContext(ctx)
    
    // Get current metrics
    currentLoad, err := r.getApplicationLoad(ctx, appOp)
    if err != nil {
        return ctrl.Result{}, err
    }
    
    // Calculate desired replicas based on load and business rules
    desiredReplicas := r.calculateOptimalReplicas(appOp, currentLoad)
    
    if desiredReplicas != appOp.Spec.Replicas {
        logger.Info("Scaling application", "current", appOp.Spec.Replicas, "desired", desiredReplicas)
        
        // Update the spec
        appOp.Spec.Replicas = desiredReplicas
        if err := r.Update(ctx, appOp); err != nil {
            return ctrl.Result{}, err
        }
        
        r.Recorder.Eventf(appOp, corev1.EventTypeNormal, "Scaling", 
            "Scaled from %d to %d replicas based on load %f", 
            appOp.Spec.Replicas, desiredReplicas, currentLoad)
    }
    
    return ctrl.Result{RequeueAfter: 30 * time.Second}, nil
}

// Custom health assessment
func (r *AppOperatorReconciler) assessApplicationHealth(ctx context.Context, appOp *mygroupv1.AppOperator) ApplicationState {
    // Check deployment status
    var deployment appsv1.Deployment
    if err := r.Get(ctx, types.NamespacedName{
        Name: appOp.Name, Namespace: appOp.Namespace,
    }, &deployment); err != nil {
        return ApplicationDegraded
    }
    
    // Custom health checks
    if deployment.Status.UnavailableReplicas > 0 {
        return ApplicationDegraded
    }
    
    // Check external dependencies if any
    if appOp.Spec.ExternalDependencies != nil {
        if !r.checkExternalDependencies(ctx, appOp) {
            return ApplicationDegraded
        }
    }
    
    return ApplicationHealthy
}

  

⚡ Key Takeaways for Production-Ready Operators

  1. Design for Resilience: Implement proper error handling, retry logic, and graceful degradation
  2. Monitor Everything: Comprehensive logging, metrics, and alerting for operator behavior
  3. Security First: Proper RBAC, network policies, and secret management
  4. Test Thoroughly: Unit tests, integration tests, and end-to-end validation
  5. Document Operations: Clear documentation for troubleshooting and day-2 operations

🔗 Integration with Modern AI/ML Workflows

Custom operators are particularly powerful for AI/ML workloads. Check out our guide on Building ML Pipelines on Kubernetes for more insights into orchestrating machine learning workflows.

For monitoring and observability, our article on Advanced Kubernetes Monitoring in 2025 provides essential patterns for tracking operator performance and application health.

❓ Frequently Asked Questions

When should I build a custom operator vs using Helm charts?
Build a custom operator when you need ongoing management of stateful applications, complex lifecycle operations, or domain-specific knowledge encoded into automation. Use Helm for simpler, stateless application deployments that don't require ongoing management.
How do I handle operator versioning and upgrades?
Implement versioned Custom Resource Definitions (CRDs), use semantic versioning for your operator, and provide migration paths for CRD schema changes. Consider using the Operator Lifecycle Manager (OLM) for managing operator deployments and upgrades.
What's the performance impact of running multiple operators?
Well-designed operators have minimal performance impact. Monitor API server load, use efficient watch configurations, implement resync periods appropriately, and consider consolidating related functionality into single operators when possible.
How do I test my custom operator effectively?
Use the envtest framework from controller-runtime for integration testing, implement comprehensive unit tests for business logic, and consider end-to-end tests with Kind clusters. Test failure scenarios and edge cases thoroughly.
Can operators work across multiple clusters?
Yes, operators can be designed for multi-cluster scenarios using tools like Cluster API, or by implementing federation patterns. However, this adds complexity around network connectivity, security, and consistency that must be carefully managed.

💬 Found this article helpful? Please leave a comment below or share it with your network to help others learn! Have you built custom operators for your applications? Share your experiences and challenges!

About LK-TECH Academy — Practical tutorials & explainers on software engineering, AI, and infrastructure. Follow for concise, hands-on guides.

Friday, 10 October 2025

Building a High-Performance API Gateway from Scratch with Golang and Gin | LK-TECH Academy

October 10, 2025 0

Building a High-Performance API Gateway from Scratch with Golang and Gin

API Gateway architecture with Golang and Gin framework showing microservices routing and load balancing

In today's microservices architecture, API gateways have become essential components that handle request routing, authentication, rate limiting, and other cross-cutting concerns. While cloud providers offer managed API gateway solutions, building your own custom gateway gives you complete control, better performance, and significant cost savings. In this comprehensive guide, we'll build a high-performance API gateway from scratch using Golang and the Gin framework, implementing enterprise-grade features like JWT authentication, rate limiting, request/response transformation, and circuit breaking.

🚀 Why Build a Custom API Gateway?

Before diving into implementation, let's understand why you might want to build a custom API gateway instead of using off-the-shelf solutions:

  • Performance Optimization: Eliminate overhead from generic solutions and tailor performance to your specific use case
  • Cost Efficiency: Avoid vendor lock-in and reduce operational costs by 60-80% compared to managed services
  • Custom Business Logic: Implement domain-specific authentication, routing, and transformation logic
  • Learning Experience: Deep understanding of distributed systems and API management principles
  • Lightweight Deployment: Minimal resource footprint compared to heavyweight enterprise API gateways

According to recent benchmarks, custom Go-based API gateways can handle up to 50,000 requests per second on a single instance, significantly outperforming many commercial solutions.

🏗️ Architecture Overview

Our API gateway will implement the following core components:

  • Request Routing: Dynamic routing based on path patterns and HTTP methods
  • Middleware Chain: Modular middleware for cross-cutting concerns
  • JWT Authentication: Secure token-based authentication
  • Rate Limiting: Request throttling per client/IP
  • Circuit Breaker: Fail-fast mechanism for unhealthy services
  • Response Transformation: Modify responses from backend services
  • Logging & Metrics: Comprehensive observability

💻 Project Setup and Basic Structure

Let's start by setting up our Go project and creating the basic structure:


// go.mod
module api-gateway

go 1.21

require (
    github.com/gin-gonic/gin v1.9.1
    github.com/golang-jwt/jwt/v5 v5.0.0
    github.com/redis/go-redis/v9 v9.0.5
    golang.org/x/time v0.3.0
)

// main.go
package main

import (
    "log"
    "os"
    
    "github.com/gin-gonic/gin"
)

func main() {
    // Initialize the gateway
    gateway := NewGateway()
    
    // Setup routes and middleware
    gateway.SetupRoutes()
    
    // Start the server
    port := os.Getenv("PORT")
    if port == "" {
        port = "8080"
    }
    
    log.Printf("API Gateway starting on port %s", port)
    if err := gateway.router.Run(":" + port); err != nil {
        log.Fatalf("Failed to start server: %v", err)
    }
}

  

🔧 Core Gateway Implementation

Now let's implement the core gateway structure with route configuration and middleware support:


// gateway.go
package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "strings"
    "time"
    
    "github.com/gin-gonic/gin"
    "github.com/redis/go-redis/v9"
)

type Gateway struct {
    router     *gin.Engine
    redis      *redis.Client
    routes     []RouteConfig
    middleware []gin.HandlerFunc
}

type RouteConfig struct {
    Path        string
    Method      string
    TargetURL   string
    Middleware  []string
    RateLimit   *RateLimitConfig
    AuthRequired bool
}

type RateLimitConfig struct {
    Requests int
    Window   time.Duration
}

func NewGateway() *Gateway {
    // Initialize Redis for rate limiting
    rdb := redis.NewClient(&redis.Options{
        Addr:     "localhost:6379",
        Password: "", // no password set
        DB:       0,  // use default DB
    })
    
    // Test Redis connection
    ctx := context.Background()
    if err := rdb.Ping(ctx).Err(); err != nil {
        log.Printf("Redis connection failed: %v", err)
    }
    
    router := gin.Default()
    
    return &Gateway{
        router: router,
        redis:  rdb,
        routes: []RouteConfig{},
        middleware: []gin.HandlerFunc{
            LoggerMiddleware(),
            RecoveryMiddleware(),
        },
    }
}

func (g *Gateway) SetupRoutes() {
    // Apply global middleware
    for _, mw := range g.middleware {
        g.router.Use(mw)
    }
    
    // Load route configurations
    g.loadRoutes()
    
    // Setup route handlers
    for _, route := range g.routes {
        g.setupRoute(route)
    }
}

func (g *Gateway) loadRoutes() {
    // In production, load from database or config file
    g.routes = []RouteConfig{
        {
            Path:        "/api/v1/users/*path",
            Method:      "ANY",
            TargetURL:   "http://user-service:8080",
            AuthRequired: true,
            RateLimit:   &RateLimitConfig{Requests: 100, Window: time.Minute},
        },
        {
            Path:        "/api/v1/products/*path",
            Method:      "ANY",
            TargetURL:   "http://product-service:8080",
            AuthRequired: true,
        },
        {
            Path:        "/auth/*path",
            Method:      "ANY",
            TargetURL:   "http://auth-service:8080",
            AuthRequired: false,
            RateLimit:   &RateLimitConfig{Requests: 10, Window: time.Minute},
        },
    }
}

func (g *Gateway) setupRoute(route RouteConfig) {
    handler := g.createProxyHandler(route)
    
    // Convert method to gin handler
    switch strings.ToUpper(route.Method) {
    case "GET":
        g.router.GET(route.Path, handler)
    case "POST":
        g.router.POST(route.Path, handler)
    case "PUT":
        g.router.PUT(route.Path, handler)
    case "DELETE":
        g.router.DELETE(route.Path, handler)
    case "ANY", "*":
        g.router.Any(route.Path, handler)
    default:
        g.router.Any(route.Path, handler)
    }
}

  

🛡️ Implementing JWT Authentication Middleware

Security is crucial for any API gateway. Let's implement JWT authentication:


// auth.go
package main

import (
    "fmt"
    "net/http"
    "strings"
    "time"
    
    "github.com/gin-gonic/gin"
    "github.com/golang-jwt/jwt/v5"
)

var jwtSecret = []byte("your-secret-key") // In production, use environment variable

type Claims struct {
    UserID string `json:"user_id"`
    Role   string `json:"role"`
    jwt.RegisteredClaims
}

func AuthMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        // Skip auth for public routes
        if !requiresAuth(c) {
            c.Next()
            return
        }
        
        authHeader := c.GetHeader("Authorization")
        if authHeader == "" {
            c.JSON(http.StatusUnauthorized, gin.H{"error": "Authorization header required"})
            c.Abort()
            return
        }
        
        // Extract token from "Bearer "
        parts := strings.Split(authHeader, " ")
        if len(parts) != 2 || parts[0] != "Bearer" {
            c.JSON(http.StatusUnauthorized, gin.H{"error": "Invalid authorization format"})
            c.Abort()
            return
        }
        
        tokenString := parts[1]
        claims, err := validateToken(tokenString)
        if err != nil {
            c.JSON(http.StatusUnauthorized, gin.H{"error": "Invalid token"})
            c.Abort()
            return
        }
        
        // Add user info to context
        c.Set("user_id", claims.UserID)
        c.Set("user_role", claims.Role)
        c.Next()
    }
}

func validateToken(tokenString string) (*Claims, error) {
    claims := &Claims{}
    
    token, err := jwt.ParseWithClaims(tokenString, claims, func(token *jwt.Token) (interface{}, error) {
        if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
            return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
        }
        return jwtSecret, nil
    })
    
    if err != nil {
        return nil, err
    }
    
    if !token.Valid {
        return nil, fmt.Errorf("invalid token")
    }
    
    return claims, nil
}

func requiresAuth(c *gin.Context) bool {
    // Check if the current route requires authentication
    // This would typically be determined by route configuration
    path := c.Request.URL.Path
    return !strings.HasPrefix(path, "/auth") && 
           !strings.HasPrefix(path, "/health") &&
           !strings.HasPrefix(path, "/docs")
}

// Generate JWT token (for auth service integration)
func GenerateToken(userID, role string) (string, error) {
    expirationTime := time.Now().Add(24 * time.Hour)
    
    claims := &Claims{
        UserID: userID,
        Role:   role,
        RegisteredClaims: jwt.RegisteredClaims{
            ExpiresAt: jwt.NewNumericDate(expirationTime),
            IssuedAt:  jwt.NewNumericDate(time.Now()),
            Issuer:    "api-gateway",
        },
    }
    
    token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
    return token.SignedString(jwtSecret)
}

  

⚡ Advanced Rate Limiting with Redis

Implement sophisticated rate limiting to protect your backend services:


// ratelimit.go
package main

import (
    "context"
    "fmt"
    "net/http"
    "strconv"
    "strings"
    "time"
    
    "github.com/gin-gonic/gin"
    "github.com/redis/go-redis/v9"
)

func RateLimitMiddleware(redisClient *redis.Client) gin.HandlerFunc {
    return func(c *gin.Context) {
        clientIP := c.ClientIP()
        path := c.Request.URL.Path
        
        // Create a unique key for this client and path
        key := fmt.Sprintf("rate_limit:%s:%s", clientIP, path)
        
        // Get current count
        ctx := context.Background()
        current, err := redisClient.Get(ctx, key).Int()
        if err != nil && err != redis.Nil {
            c.JSON(http.StatusInternalServerError, gin.H{"error": "Internal server error"})
            c.Abort()
            return
        }
        
        // Default limits - in production, configure per route
        limit := 100
        window := time.Minute
        
        // Check if exceeded
        if current >= limit {
            c.JSON(http.StatusTooManyRequests, gin.H{
                "error": "Rate limit exceeded",
                "retry_after": window.Seconds(),
            })
            c.Abort()
            return
        }
        
        // Increment counter
        pipe := redisClient.Pipeline()
        pipe.Incr(ctx, key)
        pipe.Expire(ctx, key, window)
        _, err = pipe.Exec(ctx)
        if err != nil {
            log.Printf("Redis pipeline error: %v", err)
        }
        
        // Add rate limit headers
        c.Header("X-RateLimit-Limit", strconv.Itoa(limit))
        c.Header("X-RateLimit-Remaining", strconv.Itoa(limit-current-1))
        c.Header("X-RateLimit-Reset", strconv.FormatInt(time.Now().Add(window).Unix(), 10))
        
        c.Next()
    }
}

// Token bucket implementation for more sophisticated rate limiting
type TokenBucket struct {
    redis      *redis.Client
    capacity   int
    refillRate time.Duration
}

func NewTokenBucket(redis *redis.Client, capacity int, refillRate time.Duration) *TokenBucket {
    return &TokenBucket{
        redis:      redis,
        capacity:   capacity,
        refillRate: refillRate,
    }
}

func (tb *TokenBucket) Allow(key string) (bool, error) {
    ctx := context.Background()
    
    luaScript := `
    local key = KEYS[1]
    local capacity = tonumber(ARGV[1])
    local refill_time = tonumber(ARGV[2])
    local now = tonumber(ARGV[3])
    
    local bucket = redis.call("HMGET", key, "tokens", "last_refill")
    local tokens = capacity
    local last_refill = now
    
    if bucket[1] then
        tokens = tonumber(bucket[1])
        last_refill = tonumber(bucket[2])
    end
    
    -- Calculate refill
    local time_passed = now - last_refill
    local refill_amount = math.floor(time_passed / refill_time)
    tokens = math.min(capacity, tokens + refill_amount)
    
    -- Update last refill time
    if refill_amount > 0 then
        last_refill = last_refill + (refill_amount * refill_time)
    end
    
    -- Check if request can be processed
    if tokens >= 1 then
        tokens = tokens - 1
        redis.call("HMSET", key, "tokens", tokens, "last_refill", last_refill)
        redis.call("EXPIRE", key, math.ceil(capacity * refill_time))
        return 1
    else
        return 0
    end
    `
    
    result, err := tb.redis.Eval(ctx, luaScript, []string{key}, 
        tb.capacity, 
        int64(tb.refillRate/time.Millisecond),
        time.Now().UnixMilli()).Result()
    
    if err != nil {
        return false, err
    }
    
    return result.(int64) == 1, nil
}

  

🔌 Reverse Proxy Implementation

The core of our API gateway is the reverse proxy that forwards requests to backend services:


// proxy.go
package main

import (
    "bytes"
    "io"
    "net/http"
    "net/http/httputil"
    "net/url"
    "strings"
    
    "github.com/gin-gonic/gin"
)

func (g *Gateway) createProxyHandler(route RouteConfig) gin.HandlerFunc {
    return func(c *gin.Context) {
        target, err := url.Parse(route.TargetURL)
        if err != nil {
            c.JSON(http.StatusInternalServerError, gin.H{"error": "Invalid target URL"})
            return
        }
        
        proxy := httputil.NewSingleHostReverseProxy(target)
        
        // Modify the request
        proxy.Director = func(req *http.Request) {
            req.URL.Scheme = target.Scheme
            req.URL.Host = target.Host
            req.URL.Path = singleJoiningSlash(target.Path, c.Param("path"))
            req.Host = target.Host
            
            // Preserve original headers
            if c.GetHeader("X-Real-IP") == "" {
                req.Header.Set("X-Real-IP", c.ClientIP())
            }
            req.Header.Set("X-Forwarded-For", c.ClientIP())
            req.Header.Set("X-Forwarded-Host", req.Host)
            req.Header.Set("X-Forwarded-Proto", "http") // or https in production
            
            // Add user context if authenticated
            if userID, exists := c.Get("user_id"); exists {
                req.Header.Set("X-User-ID", userID.(string))
            }
            if userRole, exists := c.Get("user_role"); exists {
                req.Header.Set("X-User-Role", userRole.(string))
            }
            
            // Remove hop-by-hop headers
            removeHopHeaders(req.Header)
        }
        
        // Modify the response
        proxy.ModifyResponse = func(resp *http.Response) error {
            // Add gateway headers
            resp.Header.Set("X-Gateway", "custom-go-gateway")
            resp.Header.Set("X-Gateway-Version", "1.0")
            return nil
        }
        
        // Error handling
        proxy.ErrorHandler = func(w http.ResponseWriter, r *http.Request, err error) {
            log.Printf("Proxy error: %v", err)
            c.JSON(http.StatusBadGateway, gin.H{
                "error": "Bad gateway",
                "message": "Unable to connect to backend service",
            })
        }
        
        // Serve the request
        proxy.ServeHTTP(c.Writer, c.Request)
    }
}

func singleJoiningSlash(a, b string) string {
    aslash := strings.HasSuffix(a, "/")
    bslash := strings.HasPrefix(b, "/")
    switch {
    case aslash && bslash:
        return a + b[1:]
    case !aslash && !bslash:
        return a + "/" + b
    }
    return a + b
}

func removeHopHeaders(header http.Header) {
    // Remove hop-by-hop headers
    hopHeaders := []string{
        "Connection",
        "Proxy-Connection",
        "Keep-Alive",
        "Proxy-Authenticate",
        "Proxy-Authorization",
        "Te",
        "Trailer",
        "Transfer-Encoding",
        "Upgrade",
    }
    
    for _, h := range hopHeaders {
        header.Del(h)
    }
}

// Request body caching for retries
type cachedBody struct {
    header http.Header
    body   []byte
}

func cacheRequestBody(c *gin.Context) (*cachedBody, error) {
    if c.Request.Body == nil {
        return &cachedBody{header: c.Request.Header.Clone()}, nil
    }
    
    body, err := io.ReadAll(c.Request.Body)
    if err != nil {
        return nil, err
    }
    
    // Restore the body for subsequent reads
    c.Request.Body = io.NopCloser(bytes.NewBuffer(body))
    
    return &cachedBody{
        header: c.Request.Header.Clone(),
        body:   body,
    }, nil
}

func (cb *cachedBody) restoreRequest(req *http.Request) {
    // Restore headers
    for key, values := range cb.header {
        for _, value := range values {
            req.Header.Add(key, value)
        }
    }
    
    // Restore body
    if cb.body != nil {
        req.Body = io.NopCloser(bytes.NewBuffer(cb.body))
        req.ContentLength = int64(len(cb.body))
    }
}

  

🚦 Circuit Breaker Pattern

Implement circuit breaking to prevent cascading failures when backend services are unhealthy:


// circuitbreaker.go
package main

import (
    "context"
    "errors"
    "sync"
    "time"
)

type CircuitState int

const (
    StateClosed CircuitState = iota
    StateOpen
    StateHalfOpen
)

type CircuitBreaker struct {
    mu sync.RWMutex
    
    name          string
    state         CircuitState
    failureCount  int
    successCount  int
    lastStateChange time.Time
    
    // Configuration
    failureThreshold int
    successThreshold int
    timeout          time.Duration
    halfOpenMaxCalls int
}

func NewCircuitBreaker(name string, failureThreshold, successThreshold int, timeout time.Duration) *CircuitBreaker {
    return &CircuitBreaker{
        name:             name,
        state:            StateClosed,
        failureThreshold: failureThreshold,
        successThreshold: successThreshold,
        timeout:          timeout,
        halfOpenMaxCalls: 1,
    }
}

func (cb *CircuitBreaker) Execute(ctx context.Context, fn func() error) error {
    if !cb.Allow() {
        return errors.New("circuit breaker is open")
    }
    
    err := fn()
    if err != nil {
        cb.onFailure()
        return err
    }
    
    cb.onSuccess()
    return nil
}

func (cb *CircuitBreaker) Allow() bool {
    cb.mu.RLock()
    defer cb.mu.RUnlock()
    
    switch cb.state {
    case StateClosed:
        return true
    case StateOpen:
        // Check if timeout has elapsed
        if time.Since(cb.lastStateChange) > cb.timeout {
            cb.mu.RUnlock()
            cb.mu.Lock()
            // Double-check after acquiring write lock
            if cb.state == StateOpen && time.Since(cb.lastStateChange) > cb.timeout {
                cb.state = StateHalfOpen
                cb.successCount = 0
                cb.lastStateChange = time.Now()
            }
            cb.mu.Unlock()
            cb.mu.RLock()
            return cb.state == StateHalfOpen
        }
        return false
    case StateHalfOpen:
        return cb.successCount < cb.halfOpenMaxCalls
    default:
        return false
    }
}

func (cb *CircuitBreaker) onSuccess() {
    cb.mu.Lock()
    defer cb.mu.Unlock()
    
    switch cb.state {
    case StateClosed:
        cb.failureCount = 0 // Reset failure count on consecutive successes
    case StateHalfOpen:
        cb.successCount++
        if cb.successCount >= cb.successThreshold {
            cb.state = StateClosed
            cb.failureCount = 0
            cb.lastStateChange = time.Now()
        }
    }
}

func (cb *CircuitBreaker) onFailure() {
    cb.mu.Lock()
    defer cb.mu.Unlock()
    
    switch cb.state {
    case StateClosed:
        cb.failureCount++
        if cb.failureCount >= cb.failureThreshold {
            cb.state = StateOpen
            cb.lastStateChange = time.Now()
        }
    case StateHalfOpen:
        cb.state = StateOpen
        cb.lastStateChange = time.Now()
    }
}

func (cb *CircuitBreaker) GetState() CircuitState {
    cb.mu.RLock()
    defer cb.mu.RUnlock()
    return cb.state
}

// Circuit breaker middleware
func CircuitBreakerMiddleware(breaker *CircuitBreaker) gin.HandlerFunc {
    return func(c *gin.Context) {
        if !breaker.Allow() {
            c.JSON(http.StatusServiceUnavailable, gin.H{
                "error": "Service temporarily unavailable",
                "circuit_breaker": "open",
            })
            c.Abort()
            return
        }
        
        // Store the original writer to capture status code
        c.Next()
        
        // Check if the request was successful
        status := c.Writer.Status()
        if status >= 500 {
            breaker.onFailure()
        } else {
            breaker.onSuccess()
        }
    }
}

  

⚡ Performance Optimization Techniques

  1. Connection Pooling: Reuse HTTP connections to backend services
  2. Response Caching: Cache frequent responses to reduce backend load
  3. Request Batching: Combine multiple requests into single backend calls
  4. Compression: Enable gzip compression for large responses
  5. Async Processing: Handle non-critical tasks asynchronously

📊 Monitoring and Observability

Add comprehensive monitoring to track gateway performance and health:


// metrics.go
package main

import (
    "time"
    
    "github.com/gin-gonic/gin"
)

type Metrics struct {
    TotalRequests    int64
    FailedRequests   int64
    AverageLatency   time.Duration
    RequestsByStatus map[int]int64
    RequestsByRoute  map[string]int64
}

func MetricsMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        start := time.Now()
        
        c.Next()
        
        duration := time.Since(start)
        status := c.Writer.Status()
        route := c.FullPath()
        
        // In production, send to metrics system (Prometheus, etc.)
        log.Printf("METRICS: method=%s path=%s status=%d duration=%v", 
            c.Request.Method, route, status, duration)
    }
}

// Health check endpoint
func (g *Gateway) setupHealthCheck() {
    g.router.GET("/health", func(c *gin.Context) {
        c.JSON(200, gin.H{
            "status":    "healthy",
            "timestamp": time.Now().Unix(),
            "version":   "1.0.0",
            "services":  g.checkBackendServices(),
        })
    })
}

func (g *Gateway) checkBackendServices() map[string]string {
    status := make(map[string]string)
    
    // Check each backend service
    for _, route := range g.routes {
        // Implementation would check if backend is reachable
        status[route.TargetURL] = "healthy" // or "unhealthy"
    }
    
    return status
}

  

❓ Frequently Asked Questions

How does this custom API gateway compare to Kong or AWS API Gateway?
Our custom Go gateway offers better performance (lower latency, higher throughput) and cost efficiency for specific use cases. While Kong and AWS API Gateway provide more features out-of-the-box, our solution gives you complete control over functionality and can be optimized for your exact requirements. For high-traffic applications, the performance gains can be significant.
Can this gateway handle WebSocket connections?
Yes, with additional implementation. The current proxy implementation works for HTTP/1.1, but WebSocket support requires upgrading the connection and handling bidirectional communication. You would need to implement WebSocket-specific proxy logic and potentially connection pooling for WebSocket sessions.
How do I add service discovery for dynamic backend services?
You can integrate with service discovery systems like Consul, etcd, or Kubernetes services. Instead of hardcoding target URLs, your gateway would query the service registry to get current backend instances and load balance between them. The RouteConfig would reference service names instead of fixed URLs.
What's the performance impact of adding multiple middleware layers?
Each middleware adds some latency, but Go's efficient goroutines and Gin's optimized middleware chain keep the overhead minimal. In benchmarks, a gateway with 5-7 middleware layers typically adds 1-3ms of latency compared to a direct connection. The key is to keep middleware logic efficient and avoid blocking operations.
How can I secure the gateway against common attacks?
Implement additional security middleware: request size limits, input validation, CORS policies, IP whitelisting, and regular security updates. Consider adding a WAF (Web Application Firewall) layer for comprehensive protection against SQL injection, XSS, and other OWASP top 10 vulnerabilities.

💬 Found this article helpful? Please leave a comment below or share it with your network to help others learn! Have you built a custom API gateway? Share your experiences and tips in the comments!

About LK-TECH Academy — Practical tutorials & explainers on software engineering, AI, and infrastructure. Follow for concise, hands-on guides.