So I understand right, there are two solutions that would have handled this before it even got to prod or at least found it in prod fast.
1. Bunch of tests that simulate exactly the scenario of signups. Hundreds of them actually inserting db records with maybe some kind of dummy stripe code.
2. Logs of the actual uuid for each person.
The second would never have been used since the tests would have caught this bug. But are important anyway.
Seems a bit rich to blame the absence of these two things on chatgpt. That’s just immature engineering practices.
Im guessing they used all those credits to set up those instances, but never took the extra step to add log ingestion or any kind of monitoring. Unique constraint violations peaking should have at least sent some kind of mail or slack notification a few hours after release (putting aside the "it didnt happen during the day because we push to prod several times daily" - which is insane in its own right).
Nothing here really sounds like GPTs fault to me. The issue is something that could easily have been done by a human and missed in PR.
1. Bunch of tests that simulate exactly the scenario of signups. Hundreds of them actually inserting db records with maybe some kind of dummy stripe code.
2. Logs of the actual uuid for each person.
The second would never have been used since the tests would have caught this bug. But are important anyway.
Seems a bit rich to blame the absence of these two things on chatgpt. That’s just immature engineering practices.