The Dark Side Of 'Vibe Coding' That We Need To Talk About

Generative AI has made it easy for users to fancy themselves as graphic designers or video producers, and now it's transforming them into software developers. Known as vibe coding, this trend is allowing people with little, if any, HTML experience to write code using GenAI exclusively, which opens obvious safety and security issues. Vibe coding might have opened the coding doors for everyone, but this trend is beginning to wear down on those either involved with the production of software or those who have to use it.

In recent months, the rise of vibe coding has begun to seep not just into the big businesses, like Microsoft or Anthropic, but also in the homegrown applications in the open-source community. With the ability to code on the fly, being able to fix or contribute to projects has never been easier, but it's also begun to cause havoc with the glut of vibe code submissions, causing several maintainers to go out of their way to prevent this from happening to their project. One of the most commonly used tools in computing, Curl, which allows for downloading via the command line, shut down its bug bounty program due to an influx of vibe-coded attempts to claim the cash. Some have even turned to "vibe hacking."

Vibe coding is bad for software development due to the potential lack of knowledge from the individual creating the code. With people trying to provide code that they potentially don't fully understand, issues that arise can't be quashed as easily. Imagine being asked to build a car, but you've only ever seen a car and never looked into how it actually functions. Once the car starts blowing out black smoke, how are you supposed to fix it? That's a boilerplate way to look at how vibe coders operate.

A report from InfoQ highlights that despite Tailwind CSS — a framework for web design — having never been more popular, documentation visits are down 40%. This means more people are using Tailwind to create HTML code, but fewer people are following up when either running into a problem or wanting to learn the nuances of the software they're using to develop with. Following this trend, Stack Overflow, which was the main resource for answering a massive range of programming questions, has seen traffic effectively vanish since ChatGPT launched.

Computer literacy is down across the board, despite us having one in our hands or pockets almost every day. What was once considered basic tasks, such as navigating a file system, are being lost on newer generations of users, who are then turning to AI to write code for programs. The key concern here is that GenAI models are designed to always attempt to provide an answer and are trained to behave in a certain fashion based on past interactions with users — to the point that if the chatbot doesn't have the answer, it will try to provide one, which is known as "AI hallucinations."

AI-generated programs made by those who don't understand what they do are increasingly dangerous, since the person behind the keyboard may not understand what they — or AI — have created. This, in turn, puts the end-user at risk due to software packages that don't behave as intended or have unknown security risks. Looking at open-source project discussions on forums, multiple conversations are about how the person providing software, like plugins for media hosting platform Jellyfin, cannot recognize the potential risk that this puts on the user. It's why, in recent weeks, Apple has decided to remove a vibe coding app from its App Store, as they cannot mitigate the dangers of anyone deploying code.

In the right hands, AI-generated code could potentially be put to great use, but multiple reports have shown that major companies, like Nvidia and Meta, have effectively stolen works from around the web to train the AI models. GenAI is built atop of others' works, which can be mainly seen through image and video generators. AI trainers harvest reams of content from the web and pour it into the model's roster of information, and then this content is used to create something when asked, without licensing or crediting the original source.

The same goes for AI code. A vast majority of the information that Claude Code can provide has been ripped straight from textbooks, forums, and other resources that had time and effort put into them. With no real way to source where the information came from, it puts open-source software at risk of take downs due to utilizing what could be stolen code.

If say, the ReactOS project, which aims to recreate Windows in an open-source method, were to ever pull from Microsoft, it would immediately nuke the project. With generative AI and vibe coding, there's often no way for users to verify where the code came from. It could get worse, as Microsoft has now enabled CoPilot training off of GitHub projects by default.