> I explicitly don't want any non-deterministic, “risk-based” or ML decision involved in a decision as to whether I can access an account which controls critical infrastructure

I don't want this on any account, infra or not. If I can supply my password and complete MFA (TOTP/hardware key) then I want access to my account. I've been in a situation where I was logging in from a known IP, provided the correct password, correct OTP, and had access to the recovery email address on my account, and I was still denied access until I authenticated using a phone number which wasn't even associated with my account. Other people aren't so lucky and simply never gain access to their account again. It's mental. Let me make the decision. Give me a button that says "Always allow access for correct password and challenge".

> There are to my recollection numerous stories of people being locked out of accounts which they have the passwords for because Google has decided that things are suspicious and having the password is not enough.

As someone who used to live/be in different countries on a regular basis, I've become so used to this that I have to remind myself that most users don't experience this regularly. I'm a bit more settled now, but I remember well the first time I saw a 'Can't complete authentication' (or something along those lines) screen, after I had already completed 1-2 extra steps. Not 'Please enable 2FA', not even 'Please use a different browser', simply 'You can't login. Have a good day.'. I've even had this on a work-related account at some point, ie one that's being paid for. Most times you can get it to work after a few days of trying your best to recreate the conditions of the last login (don't VPN to your old location, though, that usually backfires and can cost you a few more days). But I have definitely lost at least one account for good this way. I've literally started to develop anxiety about using Google login forms.

I use my own domain with an email service that I pay for (already since before this started happening) for everything important, and I can't recommend it enough. I know that you don't really "own" your domain either, but my experience with support from a local registrar is pretty good.

One example I can give is from a few years ago. I was in a car crash where my phone was destroyed. It was around 2 AM and I was about 500 meters from my office. I went there to use the stationary office phone to call my wife, but I didn't know her number. I thought I could check my Google account and tried to log in with my personal account. I had never logged in with my account at my workplace before. Of course, Google deemed this suspicious and asked me to confirm with an SMS to my destroyed phone or to open it and click "Allow" in the notification.

The only phone that was not changed in years was my mother's number, and I called her instead because this was the only phone I was able to recall.

After that I moved completely out of google and now I use Nextcloud for my contacts.

You shouldn't trust Google cloud because they'll leave you hanging out to dry. You can get yourself into a situation where your account is locked and nobody at Google will help you get back into your account. Even if you spend tens of thousands a month. There's no way to escalate your way to a resolution with a normal human being, because they care more about what they spend on support than the customer. AWS would never do that.

However, risk based security isn't the problem. Risk based security has been implemented by every major company (interested in security for its users) for the last 15 years. It has nothing to do with whether you'll be locked out of an account without recourse or whether there are alternative ways for you to log in. I used to maintain those systems using middleware. Properly implemented, they're only an inconvenience to a tiny subset of users that constantly use new devices from new locations without 2FA. And you can turn them off for specific users, add an alternative authentication method/criteria, or even fine tune the sensors.

Google's implementation is crap, and the lack of support is dangerous. But risk based security is fine.

You always have the option to turn on a third-party Identity and authentication provider and not depend on google auth at all.

For those who are worried about their accounts, I would recommend setting up Google Advanced Protection Program [1]. It will ask you for a physical security key and I didn’t come across any other checks while logging in (i.e. random checks the post is talking about).


On a slightly related note AWS just recently forced me (and probably everybody else) to separate Amazon shopping credentials from AWS credentials. I had never liked that they were common.
Most serious/"enterprise" users of GCP don't use Google's own systems, but federate Active Directory instead:

If you do so, authentication is delegated to AD. And there are other third-party options too:

Of all the reasons to choose AWS over GCP, account management would NOT be one of the ones I would consider. Amazon's account management is a fucking nightmare. I basically just don't login to Amazon anymore because it's so broken.
Using google does seem to have become a russian roulette, especially considering that the next step - reaching out for support - has also been ML-ified and one is reliant on support via hn/twitter.

Trusting google with anything that matters (email etc) is rapidly becoming problematic.

This article puts consumer accounts, enterprise accounts and what we call robot accounts in the same bag, but the reality is that the three classes of accounts have different authentication and authorisation processes and the risk evaluation is quite different.
If you are an organisation you'll have a Google Workspace account and can administer your groups and users as one would expect from any IAM solution.

Admin's having to reset passwords with Google is still rare, because Google asks users to keep their own recovery methods.

Honestly, I can understand the frustration from messing up one's private account access.

But don't blame Google, please.

Not just Google. Recently I tried to web login to my PayPal account. PayPal decided it was a risky attempt for some reason and asked me to provide a phone number for verification. So I gave it the one associated with my PayPal account for years, and it refused! Presumably because the number was a virtual one, which I used for all account registration stuff in case of switching carrier or out of cell coverage.

I was locked out of my account for several days due to the phone requirement until it clicked for me one day: maybe I could try login with the PayPal app on my iPhone? Sure enough, PayPal trusted the app and let me login to their app. Then I tried to web login again, and this time it asked me to confirm in the mobile app that it was indeed me who's attempting the web login.

Given that PayPal is handling people's money and they have good reasons to be cautious, but risk-based auth really sucks at times when you don't have enough patience for it.

I really do not understand why anybody considers Google to be a reliable provider of anything at this point.
As a former Gaia Admin, there's actually a lot more to Google Groups as the source of auth truth; In fact, I would describe it the other way around. Google Groups is an authentication system that happens to have a mailing list feature, though I'll definitely say that the UI is fairly bad for both.

I will point you, though, to the fact that Active Directory works exactly the same way, and has exactly as bad an interface for both. It's like there's a natural local maximum these products get stuck in, because there's no reasons to build two interfaces for the same product, even though it's used two very different ways.

This is not entirely unique to Google either. I have an old MSN email account I can't get into even though I know the password. It wants me to verify via an old phone number I no longer have access to.
I experienced this with my Google workspace account’s access to YouTube. I have 2FA set up using TOTP[•], it works fine. I can also log in to our work YouTube channels, but whenever I try to manage users on any channel, I get blocked. No re-auth option, nothing, just not allowed. Even from the same device and IP that I’m logged into for months. Support wasn’t able to resolve this yet.

[•] To enable TOTP I had to first add a mobile number as 2FA, then add TOTP (Google authenticator) and then remove the number (I did not want to use my personal number for work)

Related unhelpful behavior of Google's auth system:

Dropping you from an online meeting, mid-meeting, and demanding you log in again right now. Perhaps because it had been X days since you last logged in or whatever.

I don't recall experiencing it recently, but it happened a LOT in 2020 and 2021.

It's not just Google. My wife's wallet was stolen recently while we were out of town, leaving us with a single uncompromised credit card with which to get through the rest of the trip. A few days later, I got a call from the credit card company asking me to contact them. Apparently they thought there was suspicious activity on this card. I have email notifications enabled for all my cards so I knew that all the recent transactions were legit, and so I called thinking that all I would need to do was confirm that.

When I called, they started asking me a bunch of weird questions about previous addresses in order to verify my identity. I have been at the same address for 12 years. It turned out that they had contracted with a third party to do identity verification, and this third party had incorrect information. But of course they would not tell me who the third party was (I still don't know) or how I could get the mistakes fixed. I was unable to authenticate myself to their satisfaction, despite the fact that I had access to the on-line account and I could tell them the date and amount of every single transaction on the account. I was livid because I was convinced they were going to shut down the account, leaving me with no working cards several thousand miles from home.

Fortunately, they did not shut down the account. Instead, they mailed me a physical letter with a four-digit PIN, which I was able to used several weeks later to authenticate myself and get the account cleared. I never found out what happened to get the account flagged in the first place.

Since then my wife and I have reviewed our security processes and in addition to having half a dozen backup cards (no exaggeration) we now make sure that we never keep more than three of them in the same physical location in order to avoid a repeat of that situation.

I'm a little surprised there aren't more horror stories about people getting stuck in places if their wallet gets stolen on a trip. We got very, very lucky that not all of our cards were in my wife's wallet.

Rule #1 before signing up to any important service: try reaching a human being that provides actual support.
I had a similar issue with PayPal a few months ago.

I’m not sure if it was a bug or its risk-based authentication being inane. But my 2FA options were limited to SMS for several weeks, even though I had software and hardware token 2FA enabled on the account.

This seemed to have resolved itself recently and I can use all my 2FA methods again. But it was certainly annoying and I wonder if it was PayPal pushing too hard that it didn’t consider software and hardware tokens legitimate methods of verifying my identity under certain conditions.

For everyone complaining about lack of support from Google Cloud and the possibility of being locked out: make absolutely sure you have more than the free support plan otherwise even your account manager can’t do anything.

Your Google identity is supported by the GSuite or Workspace organisation and not Google Cloud. They will provide absolutely zero support unless you pay for support.

Source: close friend who works at Google Cloud who has been unable to help customers

Asking authentication service to not use risk while using phishable authentication methods is essentially asking them to not bother with any protection from phishing and various password and 2nd factor theft.

If you don't like risk-based auth, use non-phishable authentication method like security keys.

Note that false negative (letting attacker steal) is roughly equivalent as false positive (not letting actual owner access the account) from the owner's access since the owner can't access the account in both cases (most attackers try to take over the account and lock out the original user - if you are a target of sophisticated attackers who would silently steal and watch your information, you wouldn't complain about risk based protection). But the former is worse from privacy obviously, thus any sane provider tend to choose false positive over false negative. The only way out of this bind is to use non-phishable auth, and at this point, security key is about the only one, with app based notification being close.

Just something to note: This concern doesn't apply to companies that pay a lot of money and have a tam or "customer success *" person assigned to them who they can call and get things done. All the lockedout complaints are from individuals or small or medium-small (<2k) size companies. I only say this because while I don't have a single google device and haven't used a google account in years personally, I have no problem using gcp stuff for work where we have federated login plus multiple contracts with them.

When you have a contract with them, you can sue them for downtime/costs resulting from faulty lockouts. Even if they expressly made you sign that you can't, you can still have merit and your day in court with a good legal team and they know it.

From the article:

> Fundamentally, the accounts system is clearly consumer-oriented. A lot of this risk-based authentication seems based on the general premise that most users of the Google Accounts system can't really be trusted to set secure passwords and are their own worst enemies.

> This attitude... might be true for consumer accounts

I don't think this attitude is any less true for "GCP accounts". Maybe you can trust yourself to maintain password security, although, I would suggest that you might want to be a bit more humble. In any case, once you have other developers on your team, you'd be insane to trust them all to maintain password security.

Although I didn't work there for a long time now about 12ish years ago I started the risk based authentication project at Google, so if Hugo wants to blame someone specific I'm here and ready to take some arrows:

Still, I hope he'll consider a few factors.

Firstly, although I don't know exactly what they do these days back when the system was implemented adding 2FA would disable the risk based auth system and make logins purely deterministic, at least after some time had passed (account hijackers liked to add 2FA as the first step in locking down an account after taking it over, so full security didn't start immediately). So unless that's changed he can get what he wants just by using 2FA.

Secondly, if you use enterprise accounts via Workspaces then admins can turn off risk based auth or customize it by going to then Security > Authentication > Login Challenges.

Now the author rails against non-deterministic authentication, and suggests just asking users every security question every time. But there are reasons why it's not done the way he suggests. The main one is the unfortunate reality that - as he points out - some small number of users will not make it through the challenges even if they are the actual owner of the account. Therefore every challenge has a risk associated with it and the system works hard to avoid challenging logins if at all possible. It's a last resort option. Perhaps it could run some sort of training challenges which tell the user the right answers if they get them wrong, but:

a. That would make it much easier to phish the answers out of the user.

b. It would be extremely confusing to be presented with something that claims to be ID verification related but which lets you in even if you got the answers wrong. No other system works that way and it would certainly cause a non-trivial fraction of users to conclude the system was broken, or that they hadn't set it up properly (attempting to explain this won't work because a large number of people simply will not read anything put on the screen if it's in any way optional).

So that's why for a long time now all the auth systems of the big tech firms push users to provide a phone number for an account. It's not full 2FA (which can go wrong for users even worse), but risk based auth + a phone number is a dramatic upgrade in security with a very low failure rate, so they'd much rather you do that than try to train you to answer a series of quizzes.

Now why do they do it at all, why can't you opt out entirely? It's because account hacking was a major problem at the time these systems were introduced. I built out the first version of risk based auth at high speed because things were absolutely bananas back then. Account hijacking went overnight from a nearly non-existent problem to being measured in accounts hacked per second. The black market managed to start connecting people with stolen password databases, people with GPUs who could reverse hashes at high speed, people with botnets and people who could use stolen passwords for spamming/fraud to make money. Once those connections got made the quantity of abuse exploded to absurd levels e.g.

Even if you take a hardline "hacked users deserve it" attitude doing nothing wasn't an option, because the volumes of abusive mails being sent by hacked accounts was at times sufficient to get infrastructure blacklisted by third parties, so the integrity of services were being threatened for everyone. Also, many users couldn't figure out how they were getting hacked and assumed there must be a problem with Gmail.

Finally, there's this core argument:

"Such a system might have good security but it severely lacks availability. For critical infrastructure, this is a disaster."

That's, I think, pretty debatable. If you get hacked that can destroy your business far more effectively than one employee having trouble logging in until an admin can help them. Any security system can go wrong and lock out the real user, including plain old passwords, but there's been a clear trend over time for admins to insist on ever stricter security for high value accounts. Accounts with access to cloud resources are some of the highest value out there.

If you regularly change countries, you might want to rent a VPS with stable IP address and use a VPN. This way your IP address will not change and Google will not raise alarms.

Using datacenter IP address might be alarm by itself, though. So the best way is to build a route to your home router, but that might be tricky.

I don't think anyone uses Google Cloud at sticker price.

Not being able to even access the account is one risk, the product you built your business on getting abruptly sunset is another. [0]


Google has a reputation for being especially bad with this sort of thing but I tend to see it as a distinction between the underlying infrastructure being based on classic tools vs crypto. Doing things the old fashioned way you run into this kind of risk. Google turns off your account, or somebody sim swaps you for your 2FA, or you wake up to find your account deleted but you never find out why.

With crypto those types of risks systemically don't exist in the same way. You get new sets of risks like losing your private key, or someone stealing all your crypto with no recourse, or misconfiguring things so that when you try it 6 months later it doesn't work.

Both approaches have tradeoffs. Any time I see something like this though, the idea of using systems that are backed by some kind of dumb blockchain and api rather than a dumb AI filter does appeal.

The solution is to use Advanced Protection on all accounts, always.