Microsoft's Autonomous AI Agent Might Detect Malware Without Human Assistance

The arrival of generative AI software like ChatGPT prompted immediate speculation that hackers would use those programs to create and fine-tune malware attacks. Products like ChatGPT and Gemini might be great at coding, but they have guardrails in place to prevent the creation of malicious software. That said, hackers can always find novel ways to jailbreak an AI system and obtain the desired results. Using open-source AI software on a computer might also help in that regard. Plus, hackers can still create malware without AI tools.

While those concerns might be warranted, AI firms and other software companies are using AI to create protections against malware and other online scams. For example, PayPal has an AI system in place that might prevent you from sending money to an untrusted source.

Companies like Google and Microsoft will also use AI to improve their security tools, with the latter having unveiled a new AI tool that might be able to detect malware without any human assistance. On Tuesday, Microsoft unveiled Project Ire, a prototype for an autonomous AI system that can inspect and classify malware on its own.

Security researchers routinely uncover malware attacks which convince users to install software on their computers that can extract data and steal money. For example, researchers discovered a malware attack that spread on social networks like TikTok via AI-made videos where the hackers convinced unsuspecting users to install malicious software on their PCs by making them believe they were downloading genuine software to fix specific problems.

But analyzing malicious software is a resource-intensive task. Microsoft says that its Defender platform scans more than one billion devices every month. Manual review is then needed. Researchers have to carefully inspect new software, tear it apart, and understand how it works before publicizing it. The cat-and-mouse game then continues. Attackers produce new malware, and researchers try to stop them.

Project Ire could speed up malware detection tremendously if it works as intended by engineers from Microsoft Research, Microsoft Defender Research, and Microsoft Discovery & Quantum. Microsoft explained in a blog post that Project Ire "automates what is considered the gold standard in malware classification: fully reverse engineering a software file without any clues about its origin or purpose."

If an AI program can do that on its own without human supervision, the actual researchers can be freed up for additional security work that might require their attention. Also, an AI tool that can monitor for malware attacks continuously could be deployed in security programs for cloud environments and even on home computers.

Microsoft designed the autonomous AI agent to perform a sequence of steps to determine the origin and purpose of a potentially fraudulent piece of software. Project Ire will first run a triage phase where it attempts to determine the type of software the suspicious package might include and the areas of interest.

The AI will then inspect the software to reverse-engineer it using tools like angr and Ghidra. This is a key step where the AI agent reconstructs the malware's control flow graph. Project Ire will use the information from the previous steps to identify and summarize the key function of the software. The AI will also keep a log of its findings that can be inspected by human reviewers.

Project Ire will also invoke a validator tool that looks at the previous claims, the chain of evidence, and known "expert statements from malware reverse engineers on the Project Ire team." The AI agent will then create a final assessment of the software it analyzed, labeling it as malicious or benign.

"As of this writing, Project Ire has achieved a precision of 0.98 and a recall of 0.83 using public datasets of Windows drivers. It was the first reverse engineer at Microsoft, human or machine, to author a conviction case—a detection strong enough to justify automatic blocking—for a specific advanced persistent threat (APT) malware sample, which has since been identified and blocked by Microsoft Defender," Microsoft said.

The results come from a Microsoft test involving a dataset of publicly accessible Windows drivers, some of which were malicious. Project Ire identified 90% of all files correctly. It only flagged 2% of the benign files as threats.

Microsoft also ran a "more demanding test involved nearly 4,000 "hard-target" files not classified by automated systems and slated for manual review by expert reverse engineers." In this instance, the AI agent achieved a precision score of 0.89, which meant it caught nearly 9 out of 10 malicious files. However, recall was at 0.26, which indicates the autonomous AI only caught about a quarter of the total number of malware files in the sample.

These are promising results for Project Ire, indicating that it could be a valuable security tool in the future, especially after further optimization. These early successes make Project Ire suitable for use in the Microsoft Defender organization as a "Binary Analyzer for threat detection and software classification," Microsoft said. "Our goal is to scale the system's speed and accuracy so that it can correctly classify files from any source, even on first encounter. Ultimately, our vision is to detect novel malware directly in memory, at scale."