Skip to main content

EURO2024 - Attaining 800 TPS

In EURO2024 project the Project was planned to achieve high traffic throughput as of the Peak performance requirement would be seasonal by its nature. (People tends to come in and purchase postcards when the last match was about to kick).

The Challenge​

Now the first page is the most important one. We expand our audiences from just PromptPost Application to also PaoTang's application hence the first API that serve the first page is most critical.

First Page

In this API it contains only 1 API call which fetch

  1. the available team
  2. the latest postcard serials.

The data itself is quite static therefore it would be easy if we cache it via API Gateway.

Hence if it the cache is missed it will fall through to Lambda and Lambda can serve the request by query it from original sources.

Cache Missed

         +--------+         +--------+          +-----+
User --> | API Gw | ------> | Lambda |  ------> | RDS |
         +--------+         +--------+          +-----+
                     REQ               Query

Cache Hit

         +--------+
User --> | API Gw |
         +--------+

Originally we designed a Controller endpoint that offer this API called master API and cache it via API Gateway.

The endpoint designed using redis.once() type of call where it will ask Redis first if the data is available; If so, use redis. Otherwise fetch it from RDS.

const founds = await this.cache().once(CACHE_KEY_CAMPAIGN_VARIANTS(campaignId), async () => {
  const res = await DI.em.find(
    CampaignVariantEntity,
    {
      campaign: { campaignId },
    },
    {
      orderBy: {
        label: {
          th: 'ASC',
        },
      },
    },
  )
  return res.map((f) => CampaignVariantEntity.toDto(f))
})

This seems to work really well on Dev. It fast, it doesn't have complicated states to maintain Redis is merely a cache. RDS is still back up for the case of cache miss. All good!

The Problem​

However there is still a problem. Let say if Lambda took 300ms to fetch these 2 data points. a 100 requests per seconds traffic means that there will be on average 30 lambda invocation spikes per Cache Missed. And with 30 lambda invocation it will also floor the RDS proxy client connection as well. The problem seems to be a bit harder than expected.

  1. Lambda Inovocation spike.
  2. Each Lambda will increase client connections as our application like Koa doesn't know if the invocation need to use RDS or not. It will just invoke it.
  3. Lambda with Application is heavy.
  4. To make this worse. The ColdStart of Lambda will cause those 300ms to 4000ms.

So to fix this we will need a lot ligher, and faster Lambda execution.

The Solution​

What we endup doing is 3 folds.

  1. Make it light-weight - Write a new stand alone Lambda handler.
  2. Drop the RDS - new Lambda only read from Redis.
  3. Cache it on API Gateway for short period of time.

Impelmentation wise this is very simple Lambda handler, No Koa application, No RDS connection.

However we will need to update how our system treat redis. From now redis is a persistent data. When RDS changed. It will need to cascade the changes to Redis.

With all those changes; We can now end up write a new stand alone Lambda handler like so.

// cached.ts
export const handler: Handler = async function (event, context, callback) {
  context.callbackWaitsForEmptyEventLoop = false
  const awsRequestId = context.awsRequestId
  const path: string = event.path
  const method = event.method
  const prefix = `[EVT] ${awsRequestId} ${method} ${path}`
  const campaignId = `${event.headers[HEADER_KEY_CAMPAIGN_ID] || HEADER_DEFAULT_CAMPAIGN_ID}`
  console.log(prefix, 'INFO', campaignId)

  try {
    let statusCode = 200
    let result = {}
    if (path.endsWith('front-page')) {
      if (!cacheInst) {
        console.log('Initiating cache')
        cacheInst = RedisService.auto('thaipost-postcard', false)
      }
      const repo = new CachedRepository(() => cacheInst)
      result = await repo.getFrontPage(campaignId)
    } else {
      statusCode = 404
      result = {
        error: 'not found',
      }
    }
    callback(null, {
      statusCode,
      headers: { /* cors */
        'Access-Control-Allow-Origin': '*',
        'Access-Control-Allow-Methods': 'GET',
        'Access-Control-Allow-Headers': ALLOWED_HEADERS,
      },
      body: JSON.stringify({ success: statusCode === 200, data: result }),
    })
  } catch (e) {
    console.error(prefix, 'ERROR', e)
    callback(null, {
      statusCode: +(e && (e as any)).statusCode || 400,
      headers: { /* cors */
        'Access-Control-Allow-Origin': '*',
        'Access-Control-Allow-Methods': 'GET',
        'Access-Control-Allow-Headers': ALLOWED_HEADERS,
      },
      body: JSON.stringify({
        success: false,
        error: e && (e as Error).message,
      }),
    })
  }
}

Final Tip​

  • To make lambda light weight. Do not import index.ts for package to named import it. This only make sense if your JS bundle need a light weight. But for the full application like Koa that include everything in single Lambda it import from index.ts will be much easier to organized.

Example

Light weight for sure.

import { serialCode } from '../utils/serial' // this is gurantee to be lesser code bundled for sure.

Bundler if configure correctly may help you do this.

import { serialCode } from '../utils' // this will depends on TreeShaker and ESM module to do the job.