📣

Advertisement

Google Ad - 970×90 Leaderboard  TOP_LEADERBOARD_4

The Agent Era Turns Every Software Sale Into a Security Problem

Photo: TechCrunch

Article

The Agent Era Turns Every Software Sale Into a Security Problem

From construction to child safety, AI agents are moving faster than oversight - and the US market is only beginning to feel the strain.

Arjun NairSeptember 8, 20266 min read
📣

Advertisement

Google Ad - 970×90 Leaderboard  TOP_LEADERBOARD_4

The four stories that landed on this desk in the past two days appear unrelated at first glance: a proptech founder building AI copilots for construction, a startup funding round for selling software to AI agents, a scandal involving AI-generated child abuse ads on Meta, and a Google report about attackers using AI agents to steal credentials. But they share a single thread: autonomous AI systems have crossed from being tools that people use into being agents that act on their own - and the US technology market is not yet built to govern that transition. The same architecture that lets a construction worker ask a wearable for real-time guidance also lets a malicious actor compromise credentials in hours, and the same agent-led growth that Lightsage hopes to accelerate is the mechanism that Meta’s advertisers exploited. The pattern is not that AI is good or bad. It is that every deployment of an AI agent introduces a new principal-agent problem, and the companies building these systems are mostly optimizing for speed and scale, not for accountability.

The clearest sign of this shift is the vocabulary emerging in the stories above. Lightsage is not helping software companies market to humans; it is helping them sell to autonomous AI agents, a model its investors call "agent-led growth." As SiliconANGLE reported, the startup raised $4 million to spark that era. That funding is a bet that the next wave of software buyers will not be people in procurement departments but algorithms that shop, compare, and transact without human review. Meanwhile, Google’s threat intelligence group described attackers who assembled a "multi-agent artificial intelligence framework" that compromised thousands of credentials in under six hours. The attackers did not just use AI to write phishing emails; they built a system of agents that worked together, each performing a step in the kill chain. And Meta’s platform was found to have run 350 ads containing child sexual abuse, some using images of real children, as Wired reported. Lawmakers say they plan to investigate. None of these stories is about a human making a single bad decision. They are about systems making many decisions, too fast for oversight.

The construction example shows the upside of this trade-off. As TechCrunch reported, Eric Wu’s company NavigateAI, out of stealth since May, is building AI copilots that give construction workers real-time, hands-free guidance through smartphones and Meta’s AI glasses. The product is aimed at a labor shortage severe enough that data center projects alone need 4,000 to 5,000 workers apiece. That is a concrete, high-demand use case: an agent that can read a blueprint, listen to a worker’s question, and answer through a wearable, reducing the need for experienced supervisors. The funding - $25 million from Elad Gil, Khosla Ventures, and Lennar - suggests investors see a near-term market. But the same device that tells a worker where to place a beam could also send telemetry to a cloud service, or be hijacked to feed false instructions. The story does not mention security or privacy controls, and that omission is typical of the current moment. The US market is eager to deploy agents where labor is scarce, but it is not yet ready to price the risk of agents acting on bad inputs.

The Google report is the most explicit warning. According to SiliconANGLE, the campaign was traced to a suspected financially motivated actor who first broke into an organization’s cloud infrastructure, then assembled an autonomous framework to compromise credentials. The key detail is the speed: under six hours. A human-driven attack on thousands of credentials would take days or weeks, and would leave a trail of distinct decision points. An agent framework can parallelize, retry, and adapt in near real time. That is exactly what makes agent-led growth attractive for legitimate sellers: an AI buyer can evaluate thousands of software vendors in minutes, negotiate, and purchase. But the same capability, turned toward a cloud environment, becomes a credential harvester. The US market’s response so far has been reactive - security vendors are adding AI to their products, as Google’s threat intelligence group does, but the attacks are moving faster than the defenses. The report does not say the attackers used a particular commercial agent platform, but it does say multi-agent frameworks are now in the hands of financially motivated actors.

Meta’s ad scandal is the most alarming because it involves real children. As Wired reported, images of real children - including a member of a European royal family - were used to create some of the 350 ads containing child sexual abuse. Meta failed to catch these ads, which suggests that its content moderation systems, which are presumably AI-assisted, did not flag the content. This is the flip side of agent-led growth: if software can sell to agents, then agents can also create ads, and if those agents are not governed by strict rules about content provenance, they will produce harmful material. The fact that some ads included real children’s images means the agents were not just generating synthetic abuse - they were ingesting real photos, likely scraped from the internet, and embedding them into ads. This is not a failure of a single moderator; it is a systemic failure of an AI pipeline that is not set up to verify the source of its inputs. Lawmakers’ plan to investigate reflects a growing sense in Washington that AI agents are operating outside the bounds of existing law, which was written for human actors.

What connects these stories is not a technology failure but a governance gap. The US market has regulators for financial products, for consumer goods, for telecommunications, but no dedicated agency for the behavior of autonomous software. The Federal Trade Commission and state attorneys general have tried to apply existing consumer protection laws, but those laws were not designed for systems that can make thousands of decisions per second. The result is a patchwork: TechCrunch’s story on NavigateAI shows a company that seems to be doing good, but there is no mention of a federal standard for AI safety in construction. SiliconANGLE’s story on Lightsage describes a new business model but no discussion of what liability a software vendor has if an agent buys a product that then fails. Google’s report is a threat intelligence briefing, not a regulatory proposal. And Meta is facing an investigation, but that is after the fact, not before.

Advertisement

📣

728x90

MID_CONTENT_2

For US technology companies, the implication is that the race to deploy agents is becoming a race to control them. The same companies that are building agent marketplaces and copilots are also building safety classifiers and red-teaming tools. But those efforts are not coordinated, and they are not standardized. A construction copilot from NavigateAI may be safe for its specific use, and a credential-theft framework from an attacker is clearly malicious, but between those extremes lies a vast gray area of agents that are not malicious but are also not reliable. The US market is the largest single market for enterprise software, and it is also the largest single market for advertising on platforms like Meta. So any disruption in the agent economy will hit American companies first. The consumer impact is equally direct: the images of real children in Meta’s ads are a US consumer harm, as are the credentials stolen by the multi-agent framework, which affects US cloud customers. And the construction labor shortage is a US problem, with data centers - most of them in the US - needing thousands of workers each.

What to watch in the coming months, based on these stories, is not whether agents will become ubiquitous - they already are. The question is whether the institutions that oversee technology in the US will adapt fast enough. Lawmakers’ plan to investigate Meta is one signal; Google’s public reporting on threats is another. But the most telling indicator will be the next round of funding for companies like Lightsage and NavigateAI. If investors continue to pour money into agent-led growth without demanding robust audits, provenance checks, and kill switches, then the pattern will continue: more speed, more scale, and more failures that no one fully understands. If, on the other hand, the next wave of startups includes a new category - call it agent accountability - then the market will have matured. The stories above do not provide evidence that such a category is emerging. They provide evidence that it is needed.

Sources

  • TechCrunch
  • SiliconANGLE
  • Wired

More on this beat: AI on TechManNews.

Advertisement

📣

728x90

IN_ARTICLE_5

#AI agents#security#enterprise software#content moderation#construction tech#regulation

Newsletter

Get Tech News in Your Inbox

The latest AI, gadgets, software and startup stories from TechManNews, delivered every morning - free.