One of our key guiding engineering principles here at Index Exchange is building top teams and talent. To that end, we hold an internal Index Hackathon every year that brings together engineers from across the organisation to create, develop, and solve pressing technological issues facing our industry. It’s an opportunity for our engineers to demonstrate their technical prowess while learning new skills within a challenging yet fun environment.
Our most recent Index Hackathon was the first of its kind: a virtual Hackathon that saw 25 teams work for two days on different projects that drove our culture, business, and exchange upward and onward. And that’s precisely what one of the teams achieved with their winning submission.
In fact, the Index Hackathon isn’t just a team building exercise for engineers to flex their creative and technical skills; it often produces winning submissions that have a real impact on Index Exchange as a whole.
“The Hackathon at Index Exchange isn’t just a competition. In just 48 hours, we really push the limit of what’s possible not just from an innovation perspective, but from a practical, deployable perspective. And sometimes, we end up with gems that actually do get implemented on the exchange.”
Adam Harvie
Engineering lead at Index Exchange and member of one of the winning teams
As we’re gearing up for our next Index Hackathon, we thought we’d look back at how last year’s top submission has evolved into an integral tool used across the company.
Tackling a complex challenge to improve the exchange
For their Hackathon project, one of the winning teams decided to address the challenge of improving observability in the exchange. As an ad tech company, we work with media owners and buyers to process bid requests on our exchange so that media buyers can purchase ad space inventory. In fact, our exchange processes more than 220 billion bid requests a day!
Improving observability means accessing bid requests and responses to and from buy- and sell-side customers (external interfaces) and understanding how a request flows through the exchange systems (internal interfaces).
The team developed an application that links everything together in one request:
- An integration of OpenTelemtry and Jaeger into Supply and IB to instrument, generate, and collect telemetric data on every single auction
- The storage of demand and supply spans in ELK by unique traceID
- A new UI that combines both creative preview/genesis UIs to link supply/demand requests and responses
All together, the application improves observability of the requests coming in from media owners, requests going out to media buyers, and the traffic getting bid on in our increasingly complex architecture. When we run hundreds of billions of auctions per day, it’s very useful to have a tool that allows you to search for relevant auctions and sheds light on what those interactions look like. It’s become a tool for investigation, validation, and curious exploration for people across our organisation.
How a winning Index Hackathon team innovates together
The Hackathon presented an invaluable opportunity to join together our best talent and bring an idea that had been percolating among the team members to life. It’s another way we at Index help cultivate the knowledge and skills of our engineers.
This particular team embarked on their project by first determining architecture decisions and other requirements, followed by delegation of tasks. Secondly, they had to determine which technologies to use, some of which were new to the team. For example, the team was unfamiliar with Jaeger, so they needed to learn the tool and assess its usability to determine if it could help solve the issue.
After the first day, they had a functioning prototype of the UI, populated by an API, populated by Jaeger data. They then began building out features and search capabilities for the UI. The team faced some interesting challenges along the way, including randomly disappearing data (remedied by accessing Jaeger directly) and getting up to speed on certain systems including the UI and Jaeger.
While this wasn’t our usual in-person Hackathon, the team stayed connected while working remotely and regularly checked in on each other to ensure that everyone had the support they needed in a true display of teamwork and collaboration.
Following the Index Hackathon, the application was deployed to our production Kubernetes cluster and has been adopted by many people and teams across the company. The team continues to iterate and improve on it as feature requests from users come in.
It’s projects like these that truly demonstrate why engineers at Index are the best at what they do.
We can’t wait to see the submissions for our next Index Hackathon, the theme of which is Change. Accelerated. The past year has been all about adapting to a new normal, which brought us through countless changes. We embrace this change and want to accelerate it, and this year’s Hackathon will help accomplish just that.
Pitches open this October, so if you’re interested in taking part (and joining a talented team of engineers), check out our open career opportunities:
