We Are Testing The Privacy Sandbox – Here’s What We Learned So Far

We are already two months into “Mode B” testing of the Privacy Sandbox, with 3rd party cookies blocked on 1% of Chrome browsers worldwide. As part of our efforts to ensure the advertising industry retains effective tools without third party cookies, we have been actively experimenting with Privacy Sandbox tools. The results so far are encouraging, but there remains much to do.

How are we testing the Privacy Sandbox?

During this initial transition phase we have focused on:

• End-to-end testing
• Supply integrations
• Expanding supported features inside the Privacy Sandbox specification
• Campaign migration to cookieless advertising tools.

We have made good progress on all fronts. Notably, we have been able to deliver highly personalized creative into the most popular formats and placements, and our initial comparisons are indicating promising results against our legacy systems.

There have been some limitations. For example, the limited sample size of 1% has made it difficult for us to:

• Understand real supply and demand
• Train our various models on a full volume of cookieless data
• Work through the technical requirements of scaling these cookieless solutions.

These efforts require more complete participation, higher volumes of data, and more realistic economic incentives. So we are working with the Google team, our industry partners, and with regulators to ensure a responsible ramp plan that allows us and others to calibrate and tune these systems, while giving Google time to deliver all key features.

In short: we are confident in our technical implementation, and optimistic that with the right timeline and engagement from the advertising ecosystem, we will make this a success.

Our progress and initial results

We have actually been live bidding and buying in Sandbox since late 2023, and we have verified that the key building blocks of our personalized advertising solutions are functional. We can create first-party data profiles based on direct interactions between users and advertisers, and store those securely and privately via Interest Groups. We are able to access those profiles at bid time, and place smart bids; when we win the auction, we can deliver highly engaging creative with relevant product recommendations. In short, we have good technical support for more private, personalized advertising.

To date, the vast majority of our completed impressions have run via AdX or direct publisher integrations. In Q1, we worked closely with our most important supply partners to stand up and test new integrations; currently, we are receiving bid requests from six leading platforms, with eight more close to live.

Meanwhile, we are working to ensure that all of our existing clients and campaigns can run on these new rails. Our clients should note that in general this is a “silent” process – so if you haven’t heard from us, that is a good sign! Given that just 1% of the Chrome user base are using cookieless solutions, the current sample size and auction environment is too noisy to generate any useful client or campaign specific results. For the moment we are focused on enablement, and aggregate results.

What are the next steps?

We will use the remainder of the Mode B period to complete our functional testing, build out supply integrations, and fully enable our existing campaigns. It is a great opportunity for us to build a robust data set about the technical and economic performance of the technology, which we will share with the CMA mid-year. We hope to be able to tell the CMA that this system has strong support for our use cases, and a high degree of utility (especially relative to the post-cookie reality of e.g. Safari) for broader advanced advertising.

This however should not mean that we should “flip the switch” from 1% to 100%. As we have previously communicated, there are a number of crucial matters that have not, and can not, be addressed in this 1% testing phase.

We need more complete ecosystem participation and a larger sample size to properly understand real supply and demand, to iterate our various models on a full volume of cookieless data, and to work through the technical requirements of scale. In addition, we are relying on Google to address a number of outstanding issues that should be considered prerequisites for full scaling, including but not limited to:

• Confidence in on-device execution: It is imperative for Google to confirm the feasibility of these on-device operations, anticipating full ecosystem participation, without excessive reliance on timeouts and other limits.
• Support for native placements, starting with a complete specification and including enough time for supply sources to implement and test.

Our expectation is that this will require one or more intermediate ramp phases of 3-6 months in order to achieve. We will continue to work with the Google team, our industry partners and regulators to ensure a responsible ramp plan that reflects these realities.

We anticipate being able to publicly share a broader range of results once they reach statistical significance, in preparation for our submission to the CMA. Until then, we are working hard to ensure that advertisers will be able to effectively advertise on the Privacy Sandbox in the cookieless future.