Pranali Dizain Ke Mooladhar Jo Har Developer Ko Jaanna Chahiye

Har backend developer ko ek deewar se aksar takrana padta hai. Aapke laptop par application theek kaam karti hai. Teen sahayogi upabhoktaon ke saath staging mein theek kaam karti hai. Phir aap production mein deploy karte hain, ek saath hajaar log aate hain, aur database gir jaata hai. API 5xx errors dena shuru kar deti hai, frontend latak jaata hai, aur Slack mein koi vah screenshot post kar deta hai jo aapke phone ko 2 baje raat ko vibrata kar deta hai.

System design vah cheez hai jo us deewar ke doosri or rahti hai. Yah pratiroop, samjhauton, aur infrastructure nirnayon ka vah set hai jo ek khilona application ko ek production system se alag karta hai jo vastavik traffic ko bin girahe sambhal sakta hai. Achchhi khabar yah hai ki aapko senior infrastructure engineer hone ki avashyakta nahi hai mooladhar ko samajhne ke liye. Aapko lagbhag aath avdhaarnao ke baare mein jaanna hoga, ve kaise interakshan karti hain, aur pratyek ko kab upyog karna hai.

Yeh lekh system design ke un pratiroopon ko cover karta hai jo kaam karne vale backend developers ke liye sabase adhik maayne rakhte hain. Har bhag avdhaarna ko samjhata hai, ek vyavaharik udaharan dikhata hai, aur un samjhauton ka varnan karta hai jinhe aapko apnane se pahle mulyankan karna hoga. Yeh abstract textbook avdhaarnaen nahi hain — yeh upkaran hain jinko aap har baar upyog karenge jab aap kuchh aisa banate hain jise scale karna hai.

Load Balancing — Har Server Ko Vyast Rakhein, Lekin Bahut Adhik Vyast Nahi

Load balancing sabase saral aur sabase prabhavshali system design pratiroop hai jise aap implement kar sakte hain. Vichar seedha hai: saara traffic ek hi server ko bhejne ke bajay, aap aane vale anurodhon ko servers ke ek pool mein vitarit karte hain. Yah aapko ek saath do chijen deta hai: uchch uplabdhta (yadi ek server mar jata hai, doosre seva karte rahte hain) aur uchch throughput (kai servers kaam baant lete hain).

Sabase aam load balancing algorithms hain round-robin, least connections, aur IP hash. Round-robin server suchi ko kram mein ghoomta hai — saral, purvanumani, lekin is baat se anjaan ki pratyek server kitna vyast hai. Least connections pratyek anurodh ko sabse kam sakriya sambandho vale server ko bhejta hai, jo asam anurodh bhaar ko behtar sambhalta hai. IP hash client ke IP address ka upyog karke nishchayak roop se ek server chunta hai, jiska mahtva tab hota hai jab aapko sticky sessions ki avashyakta ho — yah sunishchit karna ki ek hi hamesha ek hi server se takrata ho.

Vyavahar mein, adhiktar production setups sangam ka upyog karte hain. Ek layer 4 load balancer (TCP star par kaam karta) raw connections ko reverse proxies ya API gateways ke ek pool mein vitarit karta hai, jo phir layer 7 load balancing (HTTP star par kaam karta) karte hain specific application instances tak anurodhon ko rout karne ke liye. Yah starit drishtikon data plane ko tez aur routing logic ko flexible rakhta hai.

# nginx.conf — simple round-robin load balancing across three app servers
upstream app_cluster {
    round-robin;
    server app1.internal:3000 max_fails=3 fail_timeout=30s;
    server app2.internal:3000 max_fails=3 fail_timeout=30s;
    server app3.internal:3000 max_fails=3 fail_timeout=30s;
}

server {
    listen 443 ssl;
    location / {
        proxy_pass http://app_cluster;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Adhiktar developers jo mahatvapurna vivran chhod dete hain vah health checking hai. Load balancer tabhi upyogi hai jab yah jaanata hai ki kaun se servers swasth hain. Yadi app2 crash hota hai lekin load balancer use traffic bhejta rahta hai, to aapki error rate lagbhag ek tihai se badh jaati hai. Hamesha sakriya health checks configure karein — load balancer samay-samay par pratyek server ko ping karta hai aur anukriyavihin ko pool se svachlit roop se hata deta hai.

Caching — Gati, Lekin Kis Keeemat Par?

Caching kisi bhi developer ke liye sabase adhik labhdayak performance ghatak hai. Ek akeya cache hit 200 millisecond ke database query ko 2 millisecond ke memry lookup mein badal sakta hai. Do das ki guna. Laakhon anurodhon par kiya gaya, vah antar $10,000 mahine ke database bill aur $500 ke beech ka antar hai.

Vidhi cacheng stack mein teen star hote hain, pratyek alag visheshtao ke saath. Application-star caching (in-memory caches jaise Redis ya Memcached) mehangi gananaon ya database queries ke parinamon ko store karta hai. CDN caching static aur semi-static sansadhanon ko upabhokta ke paas edge locations par store karta hai. HTTP caching Cache-Control aur ETag headers ka upyog karke browsers aur proxies ko aapke servers ko shamil kiye bina responses cache karne deta hai.

Caching ka sabase kathin bhag ise set karna nahi hai — balki cache ko invalidate karna jab antareek data badalta hai. Industry kuchh vishvasiya pratiroopon par sidd ho gayi hai. Cache-aside (jise lazy loading bhi kaha jata hai) ka arth hai ki application pahle cache check karti hai, miss hone par database par jaati hai, aur cache ko parinam se bhar deti hai. Write-through cache ka arth hai ki har write dono cache aur database ko ek saath jata hai. Write-behind cache ka arth hai ki writes pahle cache mein jate hain aur asynchronous roop se database mein flush hote hain.

Computer science mein sirf do kathin chijen hain: cache invalidation aur cheezon ka naam dena. Cache invalidation adhik kathin hai kyunki aap use sirf rename karke door nahi kar sakte.

Adhiktar applications ke liye, chhoti time-to-live ke saath cache-aside sahi default hai. Ek TTL set karein jo puraane data ke prati aapki sahanashilta se mel khata ho — user profiles ke liye 60 second, product listings ke liye 5 minute, reference data ke liye 24 ghante. Jab aapko majbooth anukoolta chahiye, write-through cache ka upyog karein lekin adhik write laatency sweekar karein. Jab aapko adhiktam read throughput chahiye aur antim anukoolta sahan kar sakte hain, to vyapak TTL ke saath cache-aside ka upyog karein.

import redis.asyncio as aioredis
import json

cache = aioredis.Redis.from_url("redis://cache:6379")

CACHE_TTL = 300  # 5 minutes

async def get_user_profile(user_id: str) -> dict:
    key = f"profile:{user_id}"
    cached = await cache.get(key)
    if cached:
        return json.loads(cached)
    profile = await db.fetch_user(user_id)
    if profile:
        await cache.setex(key, CACHE_TTL, json.dumps(profile))
    return profile

Ek chetavani: caching samasyayon ko hal karne ke bajay chhupa sakti hai. Yadi aapke database queries dheemee hain kyunki aap ek index miss kar rahe hain, to cache jodne se lakshan chhup jaata hai lekin antareek query abhi bhi dheemee hai. Cache nikaalta hai, dheemee query chalti hai, upabhokta intzaar karta hai. Hamesha pahle apne dheemee paths ko profil aur optimise karein, phir anukoolit version ke upar caching jodein.

Database Sharding — Kaam Ko Baantna Taaki Koi Ek Database Na Doobe

Sharding (kshitij prthakikaran) vah hai jiska aap upyog karte hain jab ek akeya database instance aapke write throughput ya dataset aakar ko sambhal nahi sakta. Vichar yah hai ki aapne data ko kai database instances mein baantna, jahan har instance (shard) data ka ek up-samuh rakhta hai. Application yah nirdharit karta hai ki kaun se shard se query karni hai shard key ke aadhar par — aamtaur par user ID ka hash, ek bhaugolik kshetra, ya ek samay seema.

Shard key kisi bhi sharded system mein sabase mahatvapurna nirnay hai. Ek achchhi shard key data ko shards mein samaan roop se vitarit karti hai aur aapke query pratiroopon ke saath sanjalit hoti hai. Ek buri shard key hot shards banati hai — kuchh shards jo adhiktar traffic sambhalte hain jabki baaki khali baithte hain. Udaharan ke liye, nirman timestamp se sharding kar na uchit lagta hai jab tak aapko ehsas ho ki aaj ka shard saare writes sambhalta hai jabki pichhle saal ke shard kuchh nahi sambhalte.

Sangati hash us rebalancing samasya ko hal karta hai jo naiv sharding ko satati hai. Ek saral modulo-aadharit sharding yojana (shard = hash(key) % N) mein, naya shard jodne ke liye lagbhag saare data ko punah-vyavasthit karna padta hai. Sangati hash keys aur shards dono ko ek hash ring par maan chitra kar deta hai; jab aap ek shard jodte hain, to naye shard ke turant pados ki keys ko hi sthanantarit karne ki avashyakta hoti hai. Yah upar-neeche skeyl karna adhik kam dardnaak banata hai.

• Hash-aadharit sharding — shard key ko hash karke vitarit karna; saral aur samaan lekin reshading mehanga hai
• Renge-aadharit sharding — man rainge se baantna (jaise, user ID 1–10000 shard A par, 10001–20000 shard B par); renge queries ke liye kushal lekin garm sthalon ki sambhavna
• Directory-aadharit sharding — keys ko shards se map karne ki lookup table banaye rakhna; flexible lekin lookup hop aur directory girne par ek viphalta bindu jodta hai
• Bhaugolik sharding — upabhokta kshetra se baantna; laatency ke liye utkrisht lekin musafir upabhoktaon ya global data ke liye musibat

Samjhauta jo aap sharding ke saath sweekar karte hain vah yah ki cross-shard queries mehangi ya asambhav ho jati hain. Yadi aap apne users table ko user_id se aur orders table ko user_id se shard karte hain, to pichhle 30 dinon ke saare orders ka query har shard par prahar karna hoga. Applications jinhe global analytics ya cross-shard joins ki avashyakta hoti hai, ve aksar doosre read replica (ya ek samarpit analytics database) ka upyog karti hain jo saare shards se data ko asynchronous roop se ekatrit karta hai.

CAP Pramey — Aap Sirf Do Chunn Sakte Hain

CAP pramey yeh batata hai ki ek vitarit data store ek saath teen garentiyon mein se do se adhik pradan nahi kar sakta: Consistency (har read ko sabase recent write milta hai), Availability (har anurodh ko ek pratikriya milti hai, chahe vah sabase recent data na ho), aur Partition Tolerance (pranali nodes ke beech network viphaltaon ke bavjud kaam karti rahti hai).

Vyavahar mein, partition tolerance vaikalpik nahi hai. Netvarak viphal hote hain. Packets girte hain, connections time-out hote hain, data centers ka vidyut jata hai. Toh asli chunaav CP (consistency + partition tolerance) aur AP (availability + partition tolerance) ke beech hai. CP pranali jaise etcd ya Zookeeper reads ki seva karne se inkaar kar dega yadi vah nodes mein anukoolta garenti nahi kar sakta. AP pranali jaise Cassandra ya DynamoDB jo bhi node sulabh hai, usse reads ki seva karega, chahe us node ke paas purana data ho.

Yah shaiksik antar nahi hai. Jab aap kai data centers span karne vali pranali ka dizain karte hain, to aapko nirnay karna hoga ki unke beech ka network link girne par kya hota hai. Kya aap sambhavit roop se puraane data ke saath anurodhon ki seva karte rahenge (AP)? Ya aap tab tak seva band kar denge jab tak netvarak theek nahi ho jaata (CP)? Uttar aapki application par nirbhar karta hai. Ek content delivery network ko AP hona chahiye — purana content koi content na hone se behtar hai. Ek bhugtaan prakrman pranali ko CP hona chahiye — aap kabhi kisi gahak se double charge nahi karna chahte kyunki do nodes ne partition ke dauran ek hi bhugtaan svikar kar liya.

Message Queues — Samakalik Peeda ko Asamakalik Kripa Mein Badalna

Message queues vitarit pranalion mein asamakalik prakrman ki reerh ki haddi hain. Ve ek seva (utpadak) ko upabhokta ke prakrman ki pratiksha kiye bina queue mein sandesh bhejne dete hain. Upabhokta sandesh uthata hai jab tayyar hota hai, prakrman karta hai, aur poorti sweekar karta hai. Yah utpadak ko upabhokta se samay aur sthan dono mein alag kar deta hai — unhe ek hi gati se chalne ki avashyakta nahi, ya ek hi samay par bhi.

Har gair-mamooli backend system ko kahin message queue ka upyog karna chahiye. Vidhi udaharan email bhejna hai. Jab upabhokta aapke platform par register hota hai, to aap nahi chahte ki HTTP pratikriya email delivery service ke liye pratiksha kare ki template render kare, SendGrid se jude, aur sandesh pahunchaye. Iske bajay, aapka API queue mein ek send_email event dhalti hai aur turant 201 Created pratikriya deta hai. Ek alag worker event uthata hai, email bhejta hai, aur kaary ko poora maan kar chinhit karta hai.

Do pradhan message queue models hain publish-subscribe (pub/sub) aur work queues. Pub/sub mein, har sandesh saare sadasyon ko pracharit hota hai. Yah event-sanchalit sthapatyaon ke liye upyogi hai jahan kai sewaon ko ek hi event par pratikriya karne ki avashyakta hoti hai — ek naya user registration welcome email, CRM update, aur analytics event ek saath chhhed sakta hai. Work queue mein, har sandesh theek ek upabhokta ko pahunchaya jata hai. Yah workers ke ek pool mein kaam vitarit karne ke liye upyogi hai — har image upload theek ek thumbnail generator ko jata hai.

# docker-compose.yml — minimal RabbitMQ setup for local development
version: "3.8"
services:
  rabbitmq:
    image: rabbitmq:3-management-alpine
    ports:
      - "5672:5672"   # AMQP port for producers/consumers
      - "15672:15672" # Management UI
    environment:
      RABBITMQ_DEFAULT_USER: app
      RABBITMQ_DEFAULT_PASS: dev-only-password
    volumes:
      - rabbitmq_data:/var/lib/rabbitmq

volumes:
  rabbitmq_data:

Message queues ka paida bhag viphaltaon ko saumya roop se sambhalna hai. Kya hota hai yadi upabhokta sandesh prakrman ke beech mein crash ho jata hai? RabbitMQ aur Amazon SQS ise delivery acknowledgments ke saath sambhalte hain — upabhokta ko spasht roop se sweekar karna hoga ki sandesh safaltapurvak prakrnit hua. Yadi upabhokta bina sweekar kiye dis-connect ho jata hai, sandesh punah-queue ho jata hai aur doosre upabhokta ko bhej diya jata hai. Yah kam-se-kam-ek delivery garenti ka arth hai ki aapke upabhoktaon ko idempotent hona chahiye: ek hi sandesh ko do baar prakrnit karne par vahi parinam utpanna hona chahiye.

Dead letter queues ek aur avashyak pratiroop hain. Jab ek sandesh kai prayatnon ke baad prakrnit nahi kiya ja sakta (adhah seva down hai, data virupit hai, vyavsay niyam badal gaya), to sandesh dead letter queue mein le jaya jata hai hamesha punah-prayas karne ke bajay. Ek sanchalak dead letter queue ki nigeerani karta hai, mool karan ki jaanch karta hai, aur ya sandesh theek kar ke punah-queue karta hai ya ignore karne ki pushti ke baad nikaal deta hai.

CDN aur Rate Limiting — Mukhseini Bachav

Content delivery networks aur rate limiters alag uddeshya rakhte hain lekin inmein ek samaan lakshan hai: ye aapke upabhoktaon aur aapke servers ke beech bachav ki pahli line hain. CDN static assets aur cached responses ko upabhokta ke paas rakhta hai, laatency kam karta hai aur traffic ko aapke origin servers se door pherta hai. Rate limiter kisi ek upabhokta ya client ko aapki pranali ko anurodhon se kaboo se bahar karne se rokta hai.

CDN aapki saamagri ko world ke edge servers ke global network par vitarit karke kaam karte hain. Jab Tokyo mein upabhokta ek sansadhan anurodh karta hai, CDN use nearest edge location se serve karta hai na ki Virginia mein aapke origin server tak request rout karne ke bajay. Yah laatency static assets ke liye 200 millisecond se 10 millisecond kaat deta hai. Aadhunik CDN aur aage badhte hain — ve API responses cache kar sakte hain, TLS connections samapt kar sakte hain, aur edge par serverless functions bhi execute kar sakte hain.

Rate limiting aapki pranali ko kai star par surakshit karti hai. Global rate limiting aapki puri pranali ke prati second sambhalne vale kul anurodhon ki ek seema lagati hai, traffic spikes aur DDoS aakramanon se bachav karti hai. Prati-user rate limiting yah sunishchit karti hai ki ek durabhyog kiraayadaar doosre upabhoktaon ke sansadhanon ko bhukha nahi kar sakta. Endpoint-star rate limiting alag routs par alag seemayen lagati hai — ek login endpoint 5 anurodh prati minute ki anumati de sakta hai jabki ek read-only search endpoint 100 anurodh prati minute ki anumati deta hai.

Sliding window algorithm rate limiting ke liye industry standard hai kyunki yah dono sahi aur kushal hai. Nischit antralon par counter reset karne ke bajay (jo seema par faisphod ki anumati deta hai), ek sliding window ek ghumaate hue samay window mein anurodhon par vichar karta hai. Redis ise implement karne ke liye prakritik chunaav hai — timestamps ko scores ke roop mein sorted set ka upyog karein, window ke bahar ki entries katiye, aur shesh entries giniye. Memory laagat nyuntam hai (pratyek anurodh ke kuch bytes) aur samay jटilta logarithmik hai.

// Redis-backed sliding window rate limiter (TypeScript)
import { createClient } from "redis";

const redis = createClient({ url: "redis://ratelimit:6379" });

async function checkRateLimit(
  key: string,
  limit: number,
  windowMs: number
): Promise<{ allowed: boolean; remaining: number }> {
  const now = Date.now();
  const windowStart = now - windowMs;

  const multi = redis.multi();
  multi.zRemRangeByScore(key, 0, windowStart);
  multi.zAdd(key, { score: now, value: `${now}` });
  multi.zCard(key);
  multi.expire(key, Math.ceil(windowMs / 1000));

  const [, , count] = await multi.exec() as [any, any, number];
  return {
    allowed: count <= limit,
    remaining: Math.max(0, limit - count),
  };
}

Dono CDN aur rate limiters ek mahatvapurna sanchalana siddhant saanjha karte hain: fail open ya fail closed? Yadi CDN edge origin tak nahi pahunch sakta, to kya use purana cached response serve karna chahiye (fail open) ya error vapas karna chahiye (fail closed)? Yadi rate limiter ka Redis cluster gir jaata hai, to kya sabhi anurodhon ko pass karna chahiye (fail open — overload ka jokhim) ya sabhi anurodh reject karne chahiyen (fail closed — promise bandi samay)?

Koi sarvabhaumik uttar nahi hai, lekin adhiktar pranalion ke liye achchha default hai: reads ke liye fail open, writes ke liye fail closed. Ek purana product listing page sweekary hai. Ek kho gaya kraya aadesh nahi hai. Is nirnay ko apne runbook mein spasht roop se dastavejit karein taaki on-call engineer jaan sake ki infrastructure jhukne par kya vyavhaar apekshit hai.

Sab Kuchh Ek Saath — Samjhauton Mein Sochna

System design pratiroopon ko ratna nahi hai. Yah samjhauton ko samajhna aur pahchanana hai ki kaun sa pratiroop aapki nirbandhno mein fit karta hai. Har nirnay mein anukoolta aur uplabdhta ke beech, read throughput aur write laatency ke beech, sanchalana jटilta aur raw pradarshan ke beech samjhauta hota hai. Sabase achchhe abhiyanta ve nahi hain jo sabse adhik pratiroop jante hain — ve hain jo ek samasya ko dekh sakte hain aur pahchan sakte hain ki kaun se nirbandh nishchit hain aur kaun se vartaalap yogya hain.

Yahaan ek tez nirnay framek hai jab aap ek nayi system design samasya ka samna karte hain. Apne gair-karyatmak avashyaktayon ki suchi bana kar shuru karein: apekshit traffic maatra, laatency lakshy, anukoolta avashyaktayen, infrastructure laagat ke liye budget, aur team ki technology se parichay. Phir is lekh ke pratiroopon par kaam karein aur puchhein ki kya pratyek aapko aapki avashyaktayon ke kareeb ya door le jaata hai.

Yadi laatency aapki pramukh chinta hai, to caching aur CDN se shuru karein — ve sabse kam jटilta ke liye sabse bada sudhar dete hain. Yadi uplabdhta mahatvapurna hai, to health checks ke saath load balancing ka upyog karein, apni sewaon ko state-less banaye, aur jahan vyavsay anumati de vahan AP ko CP se adhik chune. Yadi aap write-bhari kaam bhara se nipat rahe hain, to sharding aur message queues ka moolyankan jald karein — ve laakhon data rows hone se pahle parichit karana aasaan hote hain. Yadi aap ek janata API ke saath kaam kar rahe hain jise tretiye-paksh developers upabhog karenge, to din ek rate limit lagaye. Baad mein jodne ka arth hai aapki API ko version dena ya maujooda gahkon ko todna.

System design mein sabase mahatvapurna kaushalyan yah jaanana hai ki aapko kya nahi chahiye. Adhiktar applications ko sharding ki avashyakta nahi hai. Adhiktar applications ko message queue ki avashyakta nahi hai. Akal se pahle vitran system design mein sab buraiyon ki jad hai — har vitarit pranali viphalta ke aise modes prastut karti hai jo ek single-server pranali mein nahi hote. Jटilta tabhi jodein jab maapak aapko bataye, na ki pratiroop interview mein prabhavshali lagta hai.

Saral shuru karein. Sab kuchh maapein. Ek samay mein ek pratiroop jodein. Aage badhne se pahle sudhar ki pushti karein. Jo pranaliyan bachti hain, ve nahi hain jinke paas sabse adhik parishkrut sthapatya hai — ve hain jo samajhne mein aasan, sanchalit karne mein aasan, aur agli rukawat dikhne par badalne mein aasan hain.