An Introduction to Cookpad's Advertising Division in 2019

NOTE: This is a mirrored tech blog that I published in Japanese at the company blog while I was working for Cookpad in 2019.

Cookpad and Advertising

At Cookpad, advertising is a main business area with revenue second only to our Premium Services.

In the past, an article titled "What Cookpad's Ad Engineers Do" (published on 2015-11-26) was released, but since then, the technical elements, team structure, and business environment have changed significantly.

However, the following principle mentioned in that article remains unchanged:

"Cookpad's advertising has always sought to create a win-win-win situation for users, advertisers, and Cookpad itself. If ads can ultimately reach users as 'valuable information' through Cookpad, this leads to an increase in advertising unit prices."

Advertising exists to solve the challenges of companies placing ads and to deliver that value to users. What makes Cookpad's advertising business unique is that we don't develop DSP/SSP systems that support typical ad networks, but instead build our own in-house delivery system and maintain valuable user data, enabling us to create a distinctive advertising business.

I hope this article helps you better understand Cookpad's advertising business and the departments/groups that support it.

Services We Maintain and Operate

Our group maintains and operates an integrated in-house system from ad submission to delivery. While there are many detailed systems, the overview of the system architecture showing only the main components is as follows:

Here's a brief introduction to each system:

Ad Creative Admin Service

• The "submission" system
• Our business operations team inputs creatives according to orders
• Multi-functional with pure ad and network ad delivery ratio adjustment, targeting settings, real-time reporting of ad spaces, etc.

New Ads Delivery Service

• The "delivery" system (new generation)
• A system that separated the delivery logic previously embedded within the Cookpad application
• Initially a Rails application, but some functions were extracted into Go microservices to reduce infrastructure costs and improve development speed

Legacy Ads Delivery Service

• The "delivery" system (old generation)
• Initially, delivery logic was embedded within the Cookpad application itself
• Most functions have now been migrated to the new system, with almost no new development. It was kept for backward compatibility with older app versions but will be completely deprecated soon.

Machine Learning Services

• The "optimization" system
• Implemented as microservices using AWS API Gateway + Lambda to utilize Python and machine learning libraries for delivery ratio optimization and inventory prediction in the submission system implemented in Ruby/Rails

Logging (Lambda Architecture)

• Batch layer
  • The "Data Warehouse"
  • Stores all logs and is used in various parts of the submission system for delivery ratio optimization, reporting aggregation, etc.
  • Service owner is the DWH team
  • Business operations team and directors use Tableau to visualize dashboards for business purposes
• Speed layer
  • Service owner is our group
  • Real-time log infrastructure through streaming pipeline
  • Used for real-time reporting in the submission system and improving accuracy in delivery ratio optimization processing

Tracking Service

• The "Data Management Platform (DMP)"
• Service owner is the DMP team
• Provides features such as EAT (Extreme Audience Targeting), area targeting, and search keyword targeting

Technology Stack

The technology stack used in each service introduced earlier is as follows:

Technology Selection

In the previous section, I introduced our technology stack. Our team strives to select the optimal technology stack to achieve project and business results according to the evaluation axes shown in the following diagram:

We do not select technology solely based on company perspectives like "because the company pushes this technology," team perspectives like "because many people in the team are using this technology, so somehow...," or technical growth perspectives like "because I want to use this technology." We strive to make comprehensive judgments based on the following three points. Of course, we've failed in technology selection before, and this evaluation axis is not perfect, but it provides a framework for thinking.

Tech - Technical Perspective

This is an axis for evaluating the correctness of the technology itself. For example, choosing an outlandish technology for a particular challenge is not good design, no matter how difficult it is to implement. What's required is applying the appropriate technology as a "solution" to the appropriate "problem."

To do this, tech leads responsible for technology selection need to deepen their knowledge of numerous middleware and cloud services, expand their toolkit, and develop skills to purely compare and evaluate options.

Other aspects we evaluate include:

• The potential growth of the ecosystem in which the technology is developed
• The availability of tools that support development
• Global and Japanese technology trends
• The community in Japan supporting the technology

Company - Organizational Perspective

Next, we evaluate how the company as a whole relates to the technology.

For example, Cookpad is a company that heavily uses Ruby/Rails. Ruby committers work here, and there's substantial support. However, this doesn't mean that all services should be in Ruby or Rails. We often prioritize "technical perspective" and "team perspective" and choose technologies other than Ruby/Rails.

Other aspects we evaluate include:

• Compatibility with the company's mission
• How likely the technology choice will contribute to business success
• Compatibility with technologies used throughout the company (by other teams)
• Alignment with the company's engineering culture
• Advantages from a recruitment perspective

Team - Group Perspective

Finally, we evaluate from a team perspective. Specifically, we make comprehensive judgments considering the department and team's technology policies, as well as the current skill sets and desired career paths of team members from new graduates to senior members.

Specifically, we evaluate based on:

• Team members' maturity with the technology
• The learning capacity for the technology based on team members' current skill sets
• How the technology contributes to each member's career path
• Learning costs and support systems when the team scales
• Motivation for choosing the technology

Team Structure

There are four departments responsible for the "Marketing Services Domain":

Marketing Support Business Division

Responsible for planning, development, operation, and sales of domestic marketing services. Includes sales groups, a data team that verifies internal data to develop new products and create sales proposal materials, and a business operations team that supports daily submission tasks and reporting.

Marketing Planning Production Department

Responsible for planning, production, and progression of marketing services. Includes a production group that creates tie-up advertisements with companies.

Marketing Product Development Department

Responsible for planning and product development related to domestic advertising business and corporate business. Mainly comprises directors.

Media Product Development Department

Responsible for domestic corporate product development. This is the department where we belong.

Media Product Development Department

The Media Product Development Department consists of the following three groups. It's an organization with 90% engineers. I belong to the "Marketing Service Development Group," commonly known as msdev, which is responsible for the advertising domain.

Marketing Service Development Group (msdev)

Responsible for the advertising domain

Product Development Group (pdev)

Responsible for the video domain

Product Design Group

Comprises designers and directors

Collaborative Structure with the Product Development Group

The Product Development Group develops numerous services, primarily in the video domain. For recent developments, the following presentations and blogs may be helpful:

• Cookpad Video Business Development Challenges
• Cooking LIVE app "cookpadTV" has been greatly renewed!
• "cookpad storeTV" begins introduction to eight Arcs Group companies

Although we're in different groups, msdev and pdev are physically close and in the same department, so there's frequent communication. Depending on the project, members from one group may help with projects in the other group.

This provides a significant advantage of being able to access technologies used in the video domain while working in the advertising domain. For example, while handling advertising domain projects, I've also been involved in the following projects:

• Real-time log aggregation platform supporting cookpad storeTV ad delivery
• gRPC load testing

Additionally, both msdev and pdev host study sessions. Cross-group participation is, of course, possible. In the past, study sessions have covered topics such as:

• Basic Technology Deep Dives
  • TCP/IP, SQL, Rails, Go, Design Patterns, modern browser mechanisms, TypeScript
• AWS Service Deep Dives
  • DynamoDB, Parameter Store, API Gateway+Lambda, AWS IoT, CloudFront, etc.
• Service Deep Dives
  • Terraform, Stripe, Tableau Desktop, etc.
• Conference Participation Reports
  • RubyKaigi 2019
• Architecture Deep Dives of Systems We Maintain
  • Video delivery server, ad delivery server, etc.

Conclusion

The advertising domain is an exciting area with many technically challenging issues that often directly contribute to business revenue. Since we're not an ad network but rather maintain dedicated delivery servers and user data for our own business, there's a unique business aspect that offers great potential for those with interest and concern for business development.