In our new “Behind the Tech” blog series, we’re delving into the minds of our engineering team members to get a behind-the-scenes look at how they’re solving challenges in streaming TV and the ad tech world, collaborating with the industry, and making advancements to our exchange.
To kick things off, we’re looking at how our engineering team is enhancing our exchange to support streaming TV and boost efficiency and performance. I sat down with Luke Corrigall, director of engineering at Index Exchange, and vice chair of the Prebid.js committee, to learn more about the massive scale of programmatic in streaming TV, the features we’re building, and what we’re focusing on for the future.
Kylie Denk: Streaming TV has scaled rapidly, alongside growing viewership. What features have we built to help media owners and marketers realise the full value of programmatic in streaming?
Luke Corrigall: A few years ago, our entire engineering team came together to rebuild our programmatic exchange. Modernising our exchange improved efficiency and enabled us to scale and innovate across channels, including streaming TV.
We’ve also been a leader in developing new industry standards to improve programmatic in streaming TV. For example, our product and engineering leaders partner with the IAB Tech Lab and others in the industry to define and build standards like the OpenRTB 2.6 protocol. First released in 2022, it introduced new features to solve some of streaming TV’s most common challenges, such as ad podding, and we’ve since collaborated to introduce additional features into the protocol.
To dive deeper into this critical advancement, ad pods are groups of ads that run back-to-back, just like a traditional commercial break. Before OpenRTB 2.6, each ad within a pod was auctioned separately. This not only meant a massive amount of individual bid requests, but it also made it difficult to prevent frustrating viewer experiences like repetitive ads.
Now, media owners have more flexibility and control in how they construct their ad pods and can group individual ad slots into a single bid request. Buyers can send back a slate of well-informed bids without duplicates or competitive clashes. Transaction volumes are greatly reduced, boosting efficiency.
Defining an industry standard is only step one—the results come after adoption reaches a critical mass. Once the IAB Tech Lab released OpenRTB 2.6 for implementation, our teams got to work on building support for ad podding and other features. Our modernised infrastructure gave us the foundation we needed to move quickly. And as we continue to work with the IAB Tech Lab to evolve the protocol, we’ll also enhance our exchange to support new features.
KD: Every day our engineering teams are solving challenges in streaming TV and programmatic. What technologies are you using to enhance our infrastructure for efficiency and scale?
LC: Every day at Index, we send 1.3 trillion bid requests to media buyers—and we need to get responses back to our publishing partners in less than 200 milliseconds. The scale and complexity of what happens during this short window is astounding and we need the programmatic infrastructure to support it.
We use Go extensively on our back-end infrastructure as it has a good balance between speed and allowing our developers to work in a consistent environment that handles concurrency efficiently.
Working on systems at this scale and latency also requires a lot of ingenuity, analysis, and collaboration within our engineering team. We start with innovation and encourage anyone to propose a new idea and experiment with it. If it shows promise, we then share and get feedback on the approach and refine it.
This process allows us to try out new ideas quickly, but also build them the right way when they need to be run at internet scale.
KD: What impact have we seen from the advancements we’re working on?
LC: We’re laser-focused on boosting efficiency and performance. In addition to accelerating industry standards, we’ve invested in machine learning to help curate the massive scale of our exchange and made ongoing upgrades like a new hardware spec that operates at three times the throughput and half the electricity.
Our work in enabling ad podding has also led to significant efficiency gains and reductions in computing power and emissions, making a big impact on sustainability. A study we conducted with Publica and The Trade Desk found that pod bidding reduced ad selection carbon emissions by 84%.
All these advancements mean more efficiency and value for our customers. Marketers see improved performance and media owners maximise their revenue, in turn funding more content for consumers.
KD: With the streaming TV ecosystem evolving and growing quickly, what challenges are you focused on now that excite you for the future?
LC: Our teams are collaborating to figure out how we can solve the challenges of programmatic advertising in live streaming TV. With more live programming moving to streaming—sporting events, such as the Olympics, and news, particularly with the upcoming US presidential election—there’s great potential for growth in this space, but there are a lot of unanswered questions.
One question we’re asking ourselves is how to collaborate with our demand-side partners to handle the high-volume traffic surges during live events.
We’re well equipped to handle this volume quickly and efficiently at Index due to our machine learning-based traffic optimisation capabilities that we’ve been enhancing since we acquired Rivr. Additionally, we’ve continued to expand our eco-friendly data centers to handle upwards of 400 billion auctions every day, with larger numbers in sight.
The second question comes from the unpredictability of events. Ad breaks may be unplanned, such as in sports during a time out or pause in the game. Could we help media owners capture the necessary data earlier so when these opportunities do come up, we provide the best experience for marketers and viewers?
The rapid expansion in this market isn’t slowing down anytime soon. We’re excited to continue solving challenges in streaming TV and are ready to deliver innovative solutions that help push the industry forward.
If you’re passionate about solving challenges in streaming TV and beyond, we want to hear from you. Learn more about working at Index and check out our open roles.
