Notes

Smart Bidding Troubleshooter
*Internal Only*
For Search Pod
Last Update: January 22, 2018
Owner: [email protected]
go/IsThisWAI


Smart Bidding includes Target CPA, Target ROAS, Maximize Conversions, and eCPC. The comm doc for all Bid Strategies can be found here. This doc will go over what you should check when troubleshooting a smart bidding issue, and will give guidelines for what should and should not be escalated to TPEs.


Table of Contents

What to Check First
What Should NOT Be Escalated to TPEs
When Is It Working As Intended
What SHOULD Be Escalated to TPEs



What to Check First
When a smart bidding performance issue comes in, if the client is asking about a performance fluctuation, you should work through the following questions to get the full context.

What metric does the client cares about? Are they asking about a click fluctuation?
Smart bidding (tCPA, tROAS, Maximize Conversions, eCPC) only cares about conversion or conversion value related metrics. This means that other metrics may fall to the wayside if the conversion metrics are performing strongly. We generally should not troubleshoot click or impression fluctuations!

What time range is the client looking at?
We should consider two weeks of data minimum. A spike or performance drop for just one day does not merit an investigation--spikes happen, and the bid strategy algorithm will adjust and compensate over time. We always have to consider a longer time range, two weeks minimum, to see if the performance issue is sustained over a significant period. (A caveat here is that we still should not be troubleshooting quarter over quarter or year over year fluctuations, as per this article.)

Is the bid strategy portfolio or standard? Is it in limited or learning mode?
If the bid strategy is portfolio style, check the Shared Library to see the bid strategy report. If it is standard, navigate to the All Campaigns view, add in the Bid Strategy Type column, and click on the hyperlink there. That will take you to the bid strategy report. If the bid strategy is limited, then poor performance may be WAI. If it is in learning mode, we do expect performance fluctuations. To check for learning mode, you should check the change history to see if there have been a lot of changes made to the campaigns or ad groups. Changes often lead to learning mode.

What is the ad group level target CPA/ ROAS? What is the average target CPA/ROAS for the bid strategy? Was this an appropriate target CPA/ROAS to choose?
Check the ad group level CPA or ROAS targets, and be sure to consider the device bid adjustments that may be impacting the average CPA target as well. Compare these goals to the cost/conv (for tCPA) or conv value/cost (for tROAS) for the thirty days prior to the bid strategy being applied. The target they set should be aligned with this historical value for best performance. Picking a target that is significantly different from the historical value can lead to poor performance.

How far off is the actual cost/conv or conv value/cost from the target?
If they are saying that their bid strategy is not performing well and not hitting its targets, then we need to see how far off from the target it is. If it is within 15% of the target in either direction when we consider a two week time period, we would say that is WAI. (Caveat: you should still troubleshoot fully and provide best practices if they are within 15%)

Is there a conversion/conversion value delay?
The suggested conversion delay is <7 days, meaning 90% of all conversions or conversion value happens in less than 7 days. Long conversion delays can make it hard for tCPA to make predictions, so a long conversion delay could lead to poor performance. A short delay isn’t a requirement, however, since most advertisers still see good performance with long delay.

Check go/search-autobid-report for other issues that may be flagged.
This tool will show conversion/conversion value delay, changes made, what the impression share is, and any other possible red flags. It can be used to analyze tCPA, tROAS, and eCPC.

Check the impression share--is it high?
If impression share is high, this means that the campaign is already winning almost every available auction, so there is no significant headroom to increase bidding performance regardless of the bid strategy used. This means that the smart bidding strategy is providing the best possible performance, because it can bid down on auctions that are not likely to convert, which spends budget more efficiently.

Is the bid strategy doing well overall?
The last step is to consider the overall picture. When a client writes in saying that performance is bad, assess that critically. We need to look at overall performance and whether cost/conv or conv value/cost is generally converging towards the target, and if overall, we are seeing conversion or conversion value improvements.



What Should NOT Be Escalated to TPEs
The following are examples of situations that should not go to TPEs. We only send to TPEs if it seems buggy, or if performance is so far off from the 15% window that eng needs to investigate. For the below situations, we have the resources within GS to address the issue. One of these issues may mean the client is not following all of our best practices, or that it is an expected behavior for smart bidding.

The client is asking about clicks, impressions, average position, or cost per click.
As mentioned above, for smart bidding, we only care about conversion related metrics, and those are the only metrics that should be used to judge performance. Often, clicks will fluctuate or CPCs will shift when a smart bidding strategy is applied, and that is totally okay and expected.

The client set an inappropriate target to begin with.
With the wrong target, the bid strategy never has a chance to be successful. It is set up from the very beginning to fail, since it has to optimize towards an unrealistic goal.

The client has a one time spike or dip in performance.
This is an expected behavior. Unless we suspect invalid activity or that conversion tracking is broken, it is normal for smart bidding strategies to have occasional spikes or dips in overall performance.

The client is asking about performance for a specific keyword or specific day.
Specific keyword level performance should never be considered--it doesn’t make sense to do this, because the bid strategies all work to optimize at the ad group level or higher. Similarly, smart bidding doesn’t optimize individual day data--it works in the long run, so we cannot investigate a single day’s performance.

The client is looking at a period of time less than two weeks.
Smart bidding works to optimize in the long run, and two weeks is an appropriate amount of time for us to see overall performance trends emerge.

The bid strategy is in learning or limited mode.
Either of these statuses indicate that fluctuations may occur, so this is normal behavior.

There is a conversion or conversion value delay of 7 or more days.
Having a lengthy conversion or conversion value delay is a sub optimal set up, and while target CPA can work with any conversion/conversion value delay length, it will work best with shorter delays. Therefore, engineering says that we can expect some suboptimal performance or fluctuations if the delay is longer than 7 days.



When Is It Working As Intended
If a client’s escalation meets all the following criteria, it can be considered to be working as intended, and you can explain this to the client.

Performance is within 15% of the target over a sustained (two week minimum) period.
There may be one-off spikes or dips, but overall performance is steady and performance is converging towards the target.
If there is a conversion delay, performance is acceptable when we look back and exclude the delayed data.
Conversion or conversion value performance is strong or improving even though other metrics may not be.



What SHOULD Be Escalated to TPEs
This last section gives guidance on what types of cases we should be escalating to TPEs. These are cases that may give the product experts some insights into improving smart bidding, and where we suspect something is broken or not working as it should.

The client has followed all best practices for target setting.
This means the client set their targets based on historical data, and they are within 20% of the last 30 days of data.

The client is getting sufficient conversion data.
We recommend 50 conversions minimum in the last 30 days, but the minimum required is 15 in the last 30 days.


There is a minimal conversion/conversion value delay.
If the delay is less than 7 days and performance is still bad, it might be worth escalating.

There is a two or more week period of sustained, poor performance, excluding learning and limited periods.
The poor performance needs to be more than just a dip or a spike, it has to be sustained.

The client is worried about conversions or conversion value, and that concerning metric’s performance is poor.
Again, we should not escalate any cases about clicks, CPCs, etc, only cases related to conversion or conversion value metrics should get escalated.