Memo to AI: Does it hurt when we pull your plug?

Tech giants are hiring "AI welfare" specialists to tackle an increasingly surreal question: Are super-smart computers capable of suffering?

The new roles, which are popping up at firms like Google and Microsoft, sound like something out of science fiction. But as AI systems grow more sophisticated, some researchers aren't taking chances with what they see as the next frontier of digital ethics.

Anthropic, a leading AI development company, recently made waves in the tech community by hiring researcher Kyle Fish as its first AI welfare researcher. As part of the company's alignment science team, Fish's role focuses on creating frameworks for how organizations should treat AI systems ethically.

Before joining Anthropic, Fish co-authored a report examining these issues. "To be clear, our argument is not that AI systems definitely are—or will be—conscious, robustly agentic, or otherwise morally significant," the report states. Instead, the paper emphasizes the importance of preparing for various possibilities.

Fish’s report ties into the idea of artificial general intelligence (AGI) a goal of AI reaching human-level abilities—thinking, learning and even potentially feeling. OpenAI’s search for an “AI Welfare Specialist” signals a growing concern: if AI ever becomes conscious, how should we treat it?

However, IBM Consulting VP Brent Smolinski expresses skepticism about the need for AI welfare considerations at this stage.

"There's no evidence that AI is anywhere close to experiencing anything resembling consciousness," Smolinski says. "We may not even be approaching this capability with our current technological direction."

Smolinski highlights fundamental differences between human and artificial intelligence to support his position. While humans can learn language from relatively limited exposure, current AI systems require vast amounts of data—essentially the entire internet—to achieve similar capabilities. This disparity, he suggests, reveals how far current AI systems are from genuine consciousness.

The discussion centers on a question that has long challenged philosophers and computer scientists alike: what defines consciousness? Smolinski points to philosophical perspectives that define consciousness as the ability to independently affect oneself and the surrounding world. Current AI systems, he notes, operate purely by reacting to external inputs, rather than through independent agency.

Despite these reservations, the concept of AI welfare is gaining traction among major tech companies. Google DeepMind recently posted a position for research into machine consciousness, though the listing was later removed.

Researchers exploring AI consciousness propose borrowing tools from animal cognition studies to hunt for signs of machine awareness, while conceding that definitively proving artificial consciousness remains a formidable challenge.

This uncertainty creates practical challenges for the industry. Companies must balance the risk of overlooking potential ethical concerns against investing resources in protecting systems that may not require moral consideration.

Previous incidents demonstrate how easily humans can misinterpret AI capabilities. The ELIZA effect, a documented psychological phenomenon where humans attribute consciousness to machines, gained renewed attention in June 2022 when Google suspended engineer Blake Lemoine after he claimed the company's LaMDA chatbot had become sentient. Named after a 1966 computer program that simulated a therapist through pattern matching, the effect highlights how advanced language models can trigger unwarranted assumptions about machine consciousness.

Fish has acknowledged these complexities. "We don't have clear, settled takes about the core philosophical questions," he said in an interview with the AI newsletter Transformer. "But I think this could be possibly of great importance down the line, and so we're trying to make some initial progress."

In doing so, scientists face a key puzzle: how do we know if an AI is truly experiencing feelings, rather than just being very good at faking them? Today's AI can write responses that sound deeply emotional and self-aware, but that doesn't prove that models have real inner experiences. This mirrors a more profound mystery, one that even brain scientists are still trying to crack: how consciousness arises from the physical matter of our brains in the first place.

As AI technology evolves, discussions around AI welfare are likely to intensify. Yet as researchers and tech leaders debate whether AI could become conscious, critics argue we're asking the wrong questions. A 2021 paper argues that large language models (LLMs), despite their impressive abilities are just "stochastic parrots” that mimic patterns without proper understanding while often amplifying biases and harmful content from their training data.

Smolinski draws a distinction between current AI applications and AGI. "I would distinguish between AI that helps solve specific problems versus general AI," he says. While today's AI can excel at particular tasks, achieving human-like general intelligence remains a distant goal that may never be reached. "We may never even get to AGI," Smolinski says. "I recognize fully that we may not even be in the right neighborhood regarding how we're thinking about framing the underlying problem."

As for the current focus on AI welfare? "If you don't understand something," Smolinski observes, "it's always natural to be scared."