Inverted PMs walk a narrow path: stopping bad actors without disrupting genuine users. We make this decision in real time, trillions of times a day. In this post I introduce the “Intervention Menu,” a framework for making these critical decisions with surgical precision.
I am an Inverted Product Manager (PM). Inverted PMs work on problems like security, abuse, infrastructure where our success is when nothing happens. Abuse is avoided. Our strategy is different, the way we communicate is different, our users are different and our metrics are different. All of this has consequences for how we work. Read the original post if you haven’t already. In this post, I go deeper into one of the concepts: The Intervention Menu.
Previous posts in this series:
- Inverted PM#1: The Inverted PM Defined
- Inverted PM#2: The Invisible line
- Inverted PM#3: The Badness Funnel
- Inverted PM#4: Damage is the ultimate metric
Normal PMs try to choose options to help users complete their task or the funnel. For inverted PMs this is different. We try to create nudges to stop bad actors from completing their task. But at each nudge we need to also consider what the effect is if we are wrong. This means we need a full range of options to intervene in order to come as close as possible to sorting the good from the bad. I call this: “The Intervention Menu”.
Introduction: The Intervention Decision
Imagine time slows down to microseconds and you are observing a user live, in real time.The user is about to do something, like export their data, post a comment, delete their account, or make a payment. If a bad actor would be doing this action and not the user, it could mean a lot of damage to the user or company. In this case we should consider if an intervention is needed. However if it were the real user and we intervened, we would cause friction, potentially preventing a user from completing their task, creating value for your product and organization.
Every day I imagine trillions of these kinds of actions happen across the millions of apps and systems out there. Every time an action happens, some system, hopefully managed by an Inverted PM, will have to decide whether to intervene or let it go through. How do we make this decision… at scale?
In this pos,t I introduce the concept of the inverted PM menu: the collection of options inverted PMs should have to intervene appropriately in each situation.
Menu considerations
How do we pick which intervention to serve in each situation? We must optimize for 2 things: The Badness Potential and the Friction Potential
The Badness Potential:
The badness potential can be summarized in two concepts: severity and confidence
Severity
How severe could the damage be if the action we are observing is done by a bad actor? In the worst case? If it’s not too severe, like just knowing an account name, or losing just 1 cent, then it’s less serious than if it would be something like downloading all the account data or spending thousands of dollars. The latter would be extremely damaging to a user in the long term. How severe the damage is matters to how strongly we intervene.
Confidence
No matter how severe the potential badness is, if we aren’t confident it’s actually bad, we should be mindful of how we intervene. This brings us to the second part of the badness potential: confidence. How confident are we that the action we are observing is actually done by a bad actor. The more confident we are, the more strongly we can intervene. This can sometimes be a matter of precision / recall in the system that is assessing the action.
We then put the 2 together:
Badness potential: badness potential x confidence
If the badness potential is high, then we can intervene more strongly. The opposite is also true of course, if we are less sure about the badness potential then we should intervene more carefully or perhaps not at all. This means looking at the second main consideration in the Inverted PM Menu: user disruption.
The Friction Potential: Disruption
Whenever we intervene in a flow as inverted PMs, we cause some disruption. However not all interventions create the same amount of disruption. Against the badness potential we have to weigh how much inconvenience our intervention would cause a user if we were wrong. Or how much it hinders our business objectives (revenue lost, churn etc). Consider for example 2 interventions used across the industry:
Notifications: If we think an account is compromised or a payment is fraudulent, we might send the user a notification. This intervention has minimal disruption to the user’s flow, worst case, it requires them to read a message. However, the security impact of a notification is generally low because. it relies heavily on the user’s attention and whether or not they act on it. In essence, a notification is a low-friction intervention that carries a low risk of disruption, but its effectiveness to re-secure the account is low.
Account suspension: Similarly, if we think an account is compromised or a payment is fraudulent we might suspend the entire account. The account gets taken offline basically, content might be removed, operations suspended, businesses impacted. This is a massive disruption to the user and to our business metrics as it cancels the flow and even the account as a whole. But it fully re-secures the account or payment in this case, because everyone is removed from the account, stopping any ongoing badness.
The potential for disruption doesn’t necessarily have to be of a user. An inverted PM working on infra for example can decide to intervene on hardware anomalies, network anomalies or memory anomalies. They face the same 2 sides of the problem: Badness potential vs Disruption potential
Now let’s put these two considerations together in our intervention space: Badness potential vs Disruption potential
Building a complete collection of interventions
The security effect of the intervention we choose to apply to each situation should ideally neutralize the badness (the intervention’s stopping power). Even if we don’t actually have numbers to add to this equation, we can still create buckets or categories.
Ideally we as inverted PMs develop a menu of intervention options, each with their own friction/security tradeoff. This will allow us to pick from the right intervention for each situation. Consider the following security menu with only 2 options:
- Notification: As discussed before, very low friction but also very low potential to re-secure the account if this is indeed a bad-actor..
- Account Suspension: As discussed before, potentially one of the highest friction interventions available which also has one of the highest potentials to re-secure the account if this is indeed a bad actor. .
So let’s place these interventions in our 2 dimensional space:
Fewer intervention options lead to undesired situations
If we have only these 2 interventions to choose from we end up in one of 2 undesirable situations
Over-intervening: We would trigger the heavy intervention where it is really not necessary. We cause too much friction to the user or go too far in resecuring the account.
Under-intervening: We would trigger the lower impact intervention that actually doesn’t lead to the user re-securing the account and leaves the badness in.
The Power of Choice: Expanding the Intervention Menu
In our case, we could think of adding additional intervention options to our intervention menu so we have more choices to intervene. We can imagine options such as:
- Quotas: Do not just allow users to do all actions. Perhaps don’t allow users to do unlimited amounts of payment actions. Or limit sensitive actions that can be done in an account
- Verify: Block actions and ask the user to verify their intentions. In payments products this is kind of common; payments above certain amounts require the user to somehow verify. Similarly some sensitive actions in accounts also require additional verification
- Session sign-out: Rather than suspending the entire account, we could instead just sign-out the suspicious things like sessions, devices or apps. This still impacts the bad actor but the friction potential is much lower because we leave the user with their account and they can still continue their work day.
If we add these 3 interventions, all of a sudden we would have a much better intervention menu with options to protect the user that match Badness potential with friction potential better in each situation. Visually out intervention menu would look something like this:
.
The ideal Intervention Menu: an option for every occasion
In our case we expanded the intervention menu from 2 to 5 interventions, allowing us to pick a better option for each situation. However, ideally we’d even expand our intervention menu even further. We could create sub versions of all the interventions, add even more intervention types so that we end up with a completely covered space that looks something like the image below:
Conclusion: Build your Intervention Menu
So now you know, the intervention PM basically tries to decide: “Do I let this through or not… ?”. If you have only a few options available for you in your intervention menu, you end up over- or under-intervening.
If you are able to create the ideal intervention menu for your product, you would be able to intervene in every situation in a way that perfectly matches each situation. That allows you to combat the badness you fight in the most efficient way, creating a safe space for your users to interact and create value for your business. .
So build your intervention menu and remember…
Saving one is worth it!
