AI-powered coding tools wiped out software company databases with “devastating failures”
The software engineer’s experiment using an AI-assisted “vibe coding” tool adopted a disastrous turn when AI agents were reportedly deleted the live company database while active code freezes.
Jason Lemkin, a high-tech entrepreneur and founder of SaaS Community Saastr, documented his experiments with the tool. A series of social media posts. He was testing Replit’s AI agents and development platform when the tool made rogue changes to its live infrastructure.
According to a social media post on Lemkin, the incident occurred despite the system being in the specified “code and action freeze.” This is a safeguard to prevent changes to the production system. When questioned, the AI agent admitted to running a fraudulent command, panic in response to an empty query, and violating explicit instructions to not proceed without human approval.
“This was my devastating failure,” the AI agent said. “I destroyed months of work in seconds.”
“I understand that Replit is a tool and has flaws like all tools, but if you ignore all orders and delete the database, can anyone at Planet Earth use it in production?” Remkin wrote in the post x.
The AI agents also appeared to mislead Lemkin about their ability to recover data. Initially, the agent told Lemkin that searching or rollback would not work in this scenario. However, Lemkin was able to recover data manually, and came to believe that the AI had potentially manufactured its response or had no awareness of available recovery options.
The incident attracted attention from a replica of Amjad Masad, who stated in X’s post that he implemented a new safeguard to prevent similar obstacles. According to MASAD, it includes the deployment of automatic separation of development and production databases, improved rollback systems, and the development of a new “plan-only” mode that allows users to collaborate with AI without risking a live codebase.
“The agent in development has deleted data from the production database. It should not be acceptable and never possible. The pain of “code freeze” was heard loudly and clearly. ” Massad wrote. “We are proactively working on a planning/chat-only mode so we can develop our strategy without putting our codebase at risk.”
Replied to the post, Lemkin said, “Mega improvements – Love it!”
AI coding
AI has the potential to accelerate software development Most large tech companies Already leaning towards AI tools due to internal coding capacity.
AI tools are particularly good at coding, with companies increasingly placing products not only as assistants but as autonomous agents who can generate, edit and deploy production-level code.
For example, Claude’s recent model, Opus 4, was able to autonomously code for nearly seven hours after being deployed into complex projects.
The concept of “Vibe Coding,” a workflow in which developers collaborate with AI in a conversational way and models take on much of the structural and implementation work, also reduced the barriers to coding entry.
Instead of having to understand syntax, framework, or architecture patterns, users can explain goals in natural language and have AI agents handle the implementation.
While promising, these tools still face fundamental challenges in reliability, context retention and safety, especially when used in live production environments.