Pride in AI is a powerful force: It could result in doom without the oversight of collective wisdom

They have pride in their work. This is not the pride that comes from a sales award at an annual jamboree or being featured in the company newsletter, a cash reward for nudging quarterly numbers up or other such motivators. But from something deeper. This pride emanates from a felt sense that they have made a real difference to someone or to someone’s business.

In the liquor business, this might look like a salesperson who has spent years helping a distributor grow, building his territory, thinking about his margins, relationships and capital cycles with as much care as if it were her own business.

In IT services, it might be the account manager who feels genuine ownership over whether her client’s operations became more efficient. The sale is the outcome. The pride is in the effect. This distinction matters.

What is true of salespeople is equally or perhaps more true of those who work in professions whose very purpose is human well-being.

The most effective teachers carry a particular light in their eyes when they speak of a student who struggled and then found her way. The best doctors speak of a difficult diagnosis resolved or a patient restored to full life with a satisfaction that no performance bonus can replicate. Care workers who persist through the severe physical and emotional demands of this work are also often driven by this force.

Organizations that achieve sustained excellence understand this even if they don’t always articulate it this way. They build structures, processes and cultures that constantly reinforce the connection between what their people do every day and the difference it makes. They tell stories. They create visibility. They bring the salesperson face-to-face with the effect of her work.

This scaffolding matters. Pride doesn’t always emerge spontaneously; sometimes it needs to be cultivated, made legible and given form.

In this, the world of business has it harder than the social sector in one specific respect. A business is ultimately about making money. The immediate goal of the salesperson—close the deal, hit the number—is structurally at some distance from the deeper effect that generates pride. Organizations must aim to bridge that gap. The social sector, at least in principle, has no such problem. The work itself—the learning of children, the curing of patients, the well-being of communities—generates pride. It should come more naturally. Yet, with tragic regularity, it does not.

In too many parts of our public and social systems, anganwadi workers, teachers, nurses, doctors and care workers are treated as replaceable cogs in a large indifferent machine. They are blamed for systemic failures that are not of their making. Their professional judgement is second-guessed and their dignity quietly eroded. Rather than building pride in their work, these systems demolish it. This is one of the most consequential and least-discussed failures of institution-building.

But now I turn to a very different kind of pride, because pride is not always benign. It can, under certain conditions, become something dangerous. Something that licenses recklessness.

We are living through a moment when the most powerful technology companies in the world and many individuals who lead them are in the grip of a particular variety of pride. The people driving the artificial intelligence (AI) revolution are motivated by the usual things—money, market dominance and the fear of being left behind.

But there is something else that makes this moment distinct. There is a messianic quality to much of today’s discourse. A felt sense that these individuals are not merely building products but transforming the world, reshaping human civilization and bending the arc of history.

This pride is driving a breakneck pace of development that an increasing number of the very pioneers of AI—people who understand these systems better than almost anyone alive—are warning against. The risks being accumulated are together sufficient for civilizational collapse. AI could undermine economic stability as well as public safety, diminish basic human capacities for independent thought and genuine relationships, and deplete the natural world through its extraordinary consumption of energy and related carbon emissions.

There is a phrase so old that it has become a cliché. Yet, it captures something precise about this moment: pride comes before the fall.

The pride of the best salesperson, best teacher and best doctor is among the most constructive forces in the world. It should be nurtured. But the pride of those who believe their roles are too important and their mission too consequential to be constrained by ordinary caution or collective wisdom has been among the most destructive forces in human history. We would do well to tell the difference.

The author is CEO of Azim Premji Foundation.