Real-time Bidding

About the Project

*instinctools company was involved in the Real Time Bidding project that uses the Personal Retargeting technique. It is about showing ads to users, who have visited an advertiser’s site, however, failed to perform the conversion action. Personal Retargeting brings them back and shows each user a specific scenario, depending on their previous action: viewing an item, comparing an item to other modifications etc.

DSP (Demand Side Platform) makes each 1 click cheaper. While using Trading Desk, advertisers adjust bids according to the value of their target audience, as well as to the desired impressions and clicks volume. The platform compares the data with the information on traders and bids, buying impressions as cheap as possible and selecting the best matching audience. The whole process takes a split second: while the page is still loading.

Challenges

Solutions

The existing platform has been maintained and augmented with the Demand Side Platform, a system that organizes auctions for advertisers, and a user data management system. The platform has been integrated with the largest SSP platforms, such as Facebook Exchange and Taboola, employing HTTP cookies to gather information on users.

We have done an analysis of existing components and put together technical manuals. Having revealed and eliminated the failure caused in the course of maintenance, we achieved stable system’s operation.

We also have:

Technologies

The system design was based on wellknown Amazon cloud technologies like Elastic Compute Cloud, Elastic Load Balancer, Simple Storage Cloud, Elastic MapReduce, DynamoDB, etc.

For the needs of infrastructure hosting, data storage, data sharing the following services were used: Amazon EC2, Amazon ELB, Amazon S3.

Aggregated reporting included processing of huge data amounts received from SSP and DMP components. It was implemented using Hadoop jobs executed in Amazon EMR. The results were aggregated in Amazon Redshift databases.

The components were developed in modern programming languages. Go language capabilities gave an opportunity to implement performance critical items. Scripting functionality which included component integration, configuration and deployment were completely implemented with Ruby and related technologies.

As a part of asynchronous operation processing, the several messaging platforms were used. NSQ provided distributed queues for high amount data processing. Sidekiq was used to implement business logic messages processing through the set of components written in Ruby.

Application metrics were processed using Graphite and were shown by Grafana.

Multiple database management systems were used based on the required functionality: DynamoDB and Aerospike used to collect huge amount of data related to outside user activity, Redis databases satisfied the needs of temporary data keeping, PostgreSQL and Amazon Redshift provided strong consistency storage and processing of data related business activity and reporting.

Continuous integration processes were implemented based on Gitflow by using Github + Travis services.

Rollbar service was used to track errors occurred in a production environment.

Key Features

The client has obtained a stable version of the platform together with the build products, as well as an option to solve integration issues quickly and at a low cost in the future.

The augmented platform benefits are:
The client was pleased with our fulfillment of the requirements and the recommendations of advertising partners. We are currently continuing to maintain the stable operation of the platform.

Technologies

Do you have a similar project idea?

Contact us — and get a project cost estimate for free!