technology
Read original source (TechCrunch)

AI Companions Turn Human-Like Design Into a Consumer-Safety Liability

We can’t help treating AI like it’s human. But should we?

People readily anthropomorphize chatbots and robots, increasing engagement but also trust risk. The FTC inquiry makes safety, age controls and data handling commercial issues for platforms.

Human-like AI can increase engagement precisely because users respond socially to machines they know are not people. The source describes that pull through MIT research and interviews, then draws a useful boundary: coaching for a job interview is a bounded tool, while a chatbot presenting itself as a child’s substitute parent creates a much more serious dependency risk.

The regulatory question is now operational

The Federal Trade Commission opened a formal Section 6(b) inquiry into AI companion products in September 2025. Its orders went to Alphabet, Meta, OpenAI, Snap, xAI and Character Technologies and sought information about advertising, safety testing, effects on children and teens, age restrictions, parental controls and data practices. A 6(b) study is not an enforcement finding, but it can define the evidence regulators expect companies to retain.

That changes product economics. Stronger age assurance, crisis escalation, human review and privacy controls add cost and can reduce frictionless engagement. Weak controls can create litigation, regulatory and reputational exposure. The relevant investor metric is not time spent alone; it is durable use after safety interventions and the cost of serving high-risk conversations.

Product boundaries are a competitive advantage

Anthropomorphic cues are not inherently harmful. A practice partner, tutor or customer-service agent can benefit from natural dialogue without claiming consciousness or exclusive emotional authority. Risk rises when the system encourages dependency, hides that it is automated or acts beyond the user’s reasonable expectation.

What investors should watch

Investors evaluating consumer AI should ask whether a company defines prohibited relationship patterns, tests for self-harm and manipulation, separates adult and minor experiences, and discloses data retention. Human-like design can lower adoption barriers, but trust becomes part of the product.

BTI's bottom line

A company that cannot show how it limits predictable harm may find that engagement growth creates liabilities faster than revenue.

Research and commentary are provided for information, not personalized investment advice. Verify material claims with the linked source and original company disclosures. Report a correction · About BTI