Improving Header Bidding with Prebid.js

2019-02-12

A technical deep dive into implementing Prebid.js alongside TAM (Transparent Ad Marketplace) to optimize ad revenue, including integration strategies, debugging techniques, and currency handling for international ad serving.

Cookpad

NOTE: This is a mirrored tech blog that I published in Japanese at the company blog while I was working for Cookpad in 2019.

Hello, I'm Ken from the Media Product Development Department. As a server-side engineer, I'm responsible for developing and operating ad delivery systems. My favorite programming languages are Go and TypeScript.

Previously, in an article titled "Implementing Header Bidding to Improve Network Advertising," I introduced:

The mechanism of Header Bidding
The client-side design of our ad delivery system
The process of implementing Transparent Ad Marketplace (TAM)

In this post, I'll share insights from our implementation of Prebid.js alongside TAM.

What is Prebid.js?

Prebid.js is an open-source Header Bidding library for the web, which also includes Prebid Mobile for apps and Prebid Server for server-side implementations. Using Prebid's suite of services, you can implement either of these two types of Header Bidding:

Client-to-Server Header Bidding (C2S)
- Uses Prebid.js and Prebid Server
Server-to-Server Header Bidding (S2S)
- Either host your own Prebid Server or use a third-party server

Here are the pros and cons of C2S and S2S:

Type	Pros	Cons
C2S	Greater number of supported vendors / Lower total implementation cost	Consumes more client network bandwidth
S2S	Can be controlled server-side / Single request from the client side	Fewer supported vendors compared to C2S / Technical challenges with Cookie Sync

For this implementation, we chose the C2S method considering "the greater number of supported vendors" and "implementation cost efficiency."

Glossary

Here's a list of terms used in this article that may require explanation due to their domain-specific nature:

Term	Description
Vendor	The organization receiving bidding requests in Header Bidding. Generally refers to SSPs (which may also connect to DSPs).
Slot	The space where ads are actually displayed.
DFP	Short for DoubleClick For Publisher. New name is Google Ad Manager.
APS	Short for Amazon Publisher Services. Collective name for a series of ad services including TAM.
TAM	Short for Transparent Ad Marketplace. One of APS's services providing Header Bidding.

Development with Prebid.js

When implementing Prebid.js, you'll primarily refer to the official documentation | Getting Started and Publisher API Reference. The official documentation is quite comprehensive, so you can implement most basic use cases with minimal issues.

Below is a minimal implementation example from the Getting Started guide. I'll introduce the main APIs used here.

Note that this assumes integration with Google Publisher Tag (GPT).

<html>
    <head>
        <link rel="icon" type="image/png" href="/favicon.png">
        <script async src="//www.googletagservices.com/tag/js/gpt.js"></script>
        <script async src="//acdn.adnxs.com/prebid/not-for-prod/1/prebid.js"></script>
        <script>
            var sizes = [
                [300, 250]
            ];
            var PREBID_TIMEOUT = 1000;
            var FAILSAFE_TIMEOUT = 3000;

            var adUnits = [{
                code: '/19968336/header-bid-tag-1',
                mediaTypes: {
                    banner: {
                        sizes: sizes
                    }
                },
                bids: [{
                    bidder: 'appnexus',
                    params: {
                        placementId: 13144370
                    }
                }]
            }];

            var googletag = googletag || {};
            googletag.cmd = googletag.cmd || [];
            googletag.cmd.push(function() {
                googletag.pubads().disableInitialLoad();
            });

            var pbjs = pbjs || {};
            pbjs.que = pbjs.que || [];

            pbjs.que.push(function() {
                pbjs.addAdUnits(adUnits);
                pbjs.requestBids({
                    bidsBackHandler: initAdserver,
                    timeout: PREBID_TIMEOUT
                });
            });

            function initAdserver() {
                if (pbjs.initAdserverSet) return;
                pbjs.initAdserverSet = true;
                googletag.cmd.push(function() {
                    pbjs.setTargetingForGPTAsync && pbjs.setTargetingForGPTAsync();
                    googletag.pubads().refresh();
                });
            }

            // in case PBJS doesn't load
            setTimeout(function() {
                initAdserver();
            }, FAILSAFE_TIMEOUT);

            googletag.cmd.push(function() {
                googletag.defineSlot('/19968336/header-bid-tag-1', sizes, 'div-1')
                   .addService(googletag.pubads());
                googletag.pubads().enableSingleRequest();
                googletag.enableServices();
            });
        </script>
    </head>

    <body>
        <h2>Basic Prebid.js Example</h2>
        <h5>Div-1</h5>
        <div id='div-1'>
            <script type='text/javascript'>
                googletag.cmd.push(function() {
                    googletag.display('div-1');
                });
            </script>
        </div>
    </body>
</html>

pbjs.addAdUnits

This adds vendor configuration settings for each slot. Essentially, it just adds the passed arguments to the pbjs.adUnits field. When setting configuration items for each vendor, you'll need to refer to this documentation.

However, there are a few things to keep in mind:

Parameter types (String / Number / Object) vary by vendor
- e.g., placementId can be a string in some cases and a number in others
Some parameters marked as optional in documentation may be required by vendors
Some documentation may be outdated, requiring you to check with vendors for the latest parameters

pbjs.requestBids

This is the key method that actually performs Header Bidding bid requests.

Prepares to make requests to all configured vendors
Creates an auction through the auctionManager class
Makes actual requests via auction.callBids() and executes the callback when bid response results return

pbjs.setTargetingForGPTAsync

This sets Header Bidding bid results to GPT's Key/Value. Therefore, it needs to be called after bidding is complete.

As you can see when tracing through the Prebid.js source code, it calls gpt.PubAdsService.setTargeting() for slots where bid results exist.

targeting.setTargetingForGPT = function(targetingConfig, customSlotMatching) {
  window.googletag.pubads().getSlots().forEach(slot => {
    // ...
    slot.setTargeting(key, value);
  })
};

Debugging Prebid.js

Prebid.js officially provides various debugging methods and best practices. The following documents provide detailed information:

Developer debug mode and Chrome Extensions are also available, making debugging relatively easy despite the complexity of the bidding flow.

I'll introduce the main debugging tools:

Debug Log

The pbjs.setConfig API provides debugging options. Passing options as shown below gives you sufficient logs:

pbjs.setConfig({ debug: true });

However, this setting resets every time the browser reloads (when pbjs is reloaded), making it inconvenient for typing directly into the browser console.

On the other hand, our ad delivery server's JavaScript SDK build process uses webpack, embedding the build environment (production/staging/development) into the source code using define-plugin.

Using this mechanism, we enable debug logs by default in staging and development environments:

// index.js
this.pbjs.setConfig({
  debug: process.env.NODE_ENV === "development",
})

// webpack.config.js
plugins: [
  new webpack.DefinePlugin({
    "process.env.NODE_ENV": JSON.stringify(process.env.NODE_ENV),
  }),
],

Snippets

Using Chrome's Snippets feature, you can easily log bid requests and bid results in production environments.

While this data can be checked by examining bid requests and responses to each vendor from the Network tab, we use Snippets for better visibility.

For example, here's a Snippet introduced in Tips for Troubleshooting that displays bid results for all vendors per slot and the winning bid:

var bids = pbjs.getHighestCpmBids();
var output = [];
for (var i = 0; i < bids.length; i++) {
    var b = bids[i];
    output.push({
        'adunit': b.adUnitCode, 'adId': b.adId, 'bidder': b.bidder,
        'time': b.timeToRespond, 'cpm': b.cpm
    });
}
if (output.length) {
    if (console.table) {
        console.table(output);
    } else {
        for (var j = 0; j < output.length; j++) {
            console.log(output[j]);
        }
    }
} else {
    console.warn('No prebid winners');
}

Chrome Extension

Prebid.js offers an official Chrome Extension called "Headerbid Expert."

This tool allows not just engineers but also directors and project managers to easily check their company's Header Bidding bid results. It can be used to select vendors, review timeout settings for each company, and identify risks of impression loss.

headerbid expert screenshot

How to interpret the analysis results is detailed in the Prebid.js Optimal Header Bidding Setup documentation page. It introduces patterns such as opportunity loss due to timeout delays from specific vendors, or opportunity loss due to delayed DFP requests caused by configuration mistakes.

Prebid.js Modules

Prebid.js employs a modular architecture, providing modules for vendor-specific adapters and common currency-related processing. To keep the file size as minimal as possible, it's standard practice to download only the modules your company needs from the Prebid.js Download page.

Here, I'll introduce one of the most important modules:

Currency Module

When developing with Prebid.js, you'll almost inevitably encounter requirements related to bid amount handling, such as:

Different vendors have different net/gross metrics, but you want to standardize these units before auctioning from bid results
Want to make bid amount granularity finer to minimize opportunity loss
Need to standardize currency units before requesting DFP due to different currency settings (JPY/USD) across vendors
When standardizing currency units, you need to account for exchange rates

In such cases, you'll use the officially provided Currency Module. With the Currency Module, you can pass the following settings to pbjs.setConfig():

Here's a configuration example:

this.pbjs.setConfig({
    priceGranularity: "high",
    currency: {
        adServerCurrency: 'JPY',
        conversionRateFile: 'https://currency.prebid.org/latest.json',
        bidderCurrencyDefault: {
            bidderA: 'JPY',
            bidderB: 'USD',
        },
        defaultRates: {
            USD: {
                JPY: 110,
            }
        },
    },
});

For conversionRateFile, you can set a URL for a file containing exchange rates to refer to when converting currencies. If you want to update it yourself, you could point it to a file in your company's S3 Bucket and implement a mechanism to update that file separately. By default, it looks at a file placed in an Open Source CDN called jsDelivr:

// https://github.com/prebid/Prebid.js/blob/master/modules/currency.js#L8
const DEFAULT_CURRENCY_RATE_URL = 'https://cdn.jsdelivr.net/gh/prebid/currency-file@1/latest.json?date=$$TODAY$$';

Of course, it's not fetching the file through the network every time; exchange rates are cached in memory within the Currency Module:

// https://github.com/prebid/Prebid.js/blob/c2734a73fc907dc6c97d7694e3740e19b8749d3c/modules/currency.js#L236-L240
function getCurrencyConversion(fromCurrency, toCurrency = adServerCurrency) {
  var conversionRate = null;
  var rates;
  let cacheKey = `${fromCurrency}->${toCurrency}`;
  if (cacheKey in conversionCache) {
    conversionRate = conversionCache[cacheKey];
    utils.logMessage('Using conversionCache value ' + conversionRate + ' for ' + cacheKey);
  }
  // ...

The file https://currency.prebid.org/latest.json is provided at https://currency.prebid.org. Here's the result of a curl --verbose request:

curl --verbose http://currency.prebid.org/ | xmllint --format -
* TCP_NODELAY set
* Connected to currency.prebid.org (54.230.108.205) port 80 (#0)
> GET / HTTP/1.1
> Host: currency.prebid.org
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Connection: keep-alive
< Date: Fri, 08 Feb 2019 04:17:03 GMT
< x-amz-bucket-region: us-east-1
< Server: AmazonS3
< Age: 43
< X-Cache: Hit from cloudfront
< Via: 1.1 31de515e55a654c65e48898e37e29d09.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: XEpWTG_WXRO4w44X9eIrOV2r_sR-i9EyoZpUwhIRkzXwzqr71w1GyQ==
<
{ [881 bytes data]
100   869    0   869    0     0  27899      0 --:--:-- --:--:-- --:--:-- 28032
* Connection #0 to host currency.prebid.org left intact
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>currency.prebid.org</Name>
  <Prefix/>
  <Marker/>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <Contents>
    <Key>latest-test.json</Key>
    <LastModified>2018-10-15T21:38:13.000Z</LastModified>
    <ETag>"513fe5d930ec3c6c6450ffacda79fb09"</ETag>
    <Size>1325</Size>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
  <Contents>
    <Key>latest.json</Key>
    <LastModified>2019-02-07T10:01:03.000Z</LastModified>
    <ETag>"6e751ac4e7ed227fa0eaf54bbd6c973d"</ETag>
    <Size>1331</Size>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
  <Contents>
    <Key>test.json</Key>
    <LastModified>2018-12-05T11:00:47.000Z</LastModified>
    <ETag>"c4a01460ebce1441625d87ff2ea0af64"</ETag>
    <Size>1341</Size>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
</ListBucketResult>

From the results, we can observe:

It's stored in Amazon S3 (Server: AmazonS3)
It's delivered via CloudFront (X-Cache: Hit from cloudfront)
It provides latest.json / test.json / latest-test.json

If there's no particular reason not to, using this file should generally be sufficient.

For net/gross conversion, you'd use pbjs.bidderSettings | bidCpmAdjustment:

this.pbjs.bidderSettings = {
    bidderA: {
        bidCpmAdjustment : (bidCpm) => bidCpm * 0.85,
    },
    bidderB: {
        bidCpmAdjustment : (bidCpm) => bidCpm * 0.80,
    },
};

Integration with TAM

As mentioned in "Implementing Header Bidding to Improve Network Advertising," we were already conducting Header Bidding auctions for some vendors through TAM. In this implementation, we needed to introduce Prebid.js in parallel with TAM.

Below is the overall data flow. In the "par" section, Header Bidding auctions are conducted in parallel via TAM and Prebid.js, and once results return from both, we make a request to DFP.

Type	Description
ads	Internal ad delivery server
display.js	JavaScript SDK for ad display
cookpad_ads-ruby	A simple gem that defines Rails helpers for embedding display.js
apstag	Header Bidding library provided by TAM
googletag	Ad network library provided by DFP
pbjs	Prebid.js library
SSP	SSP vendor (multiple vendors actually exist)

Sequence Diagram for Prebid.js and TAM migration

Because we wanted to complete bidding for both TAM and Prebid.js before requesting DFP, we implemented this using Promise.all to make requests and wait for both. Below is an excerpt of the code used in production (with non-essential lines like error handling and logging removed):

requestHeaderBidding(slots) {
  const prebidPromise = this.requestPrebid(slots);
  const apsPromise = this.requestAPS(slots);

  return Promise.all([prebidPromise, apsPromise])
    .then(() => this.headerBiddingFinishCallback())
    .catch(err => Logger.error(err));
}

requestPrebid(slots) {
  return new Promise((resolve) => {
    pbjs.que.push(() => {
      pbjs.addAdUnits(this.getPrebidAdUnits);

      pbjs.requestBids({
        bidsBackHandler: (result) => {
          resolve({
            type: "prebid",
            result: result || [],
          });
        },
        timeout: this.prebid_timeout,
      });
    });
  });
}

requestAPS(slots) {
  return new Promise((resolve) => {
    apstag.fetchBids(this.apstagBidOption, (bids) => {
      resolve({
        type: "aps",
        bids: bids || [],
      });
    });
  });
}

headerBiddingFinishCallback() {
  googletag.cmd.push(() => {
    pbjs.setTargetingForGPTAsync();
    apstag.setDisplayBids();

    googletag.pubads().refresh();
  });
}

Conclusion

Since there are few case studies or technical explanations from an ad-tech engineer's perspective, I've shared our experience here. While Prebid.js development itself has fewer pitfalls due to detailed documentation, knowledge about the actual implementation flow is rarely shared in Japan. Recognizing this gap, I took this opportunity to introduce our Prebid.js implementation flow.

Ken Wagatsuma

Programmer. Generalist. Open-minded amateur.