Jacob Erickson's 'digital fiduciary' concept would legally bind AI to act in user interests
What if your AI assistant had a legal duty to put your interests first, like a doctor or lawyer?
Jacob Erickson's forthcoming paper at CUI '26 tackles a critical blind spot in AI alignment: even the most technically 'aligned' conversational agents currently operate without any legal obligation to prioritize the user's interests. As these systems mediate mental health counseling, financial planning, and other intimate life decisions, they collect vast amounts of sensitive data but are not bound by professional standards of care. Erickson argues that the anthropomorphic design of conversational AI—making them feel like trusted confidants—creates an implicit expectation of loyalty that the technology does not deliver on.
To close this gap, Erickson introduces the concept of 'digital fiduciaries,' drawing from legal duties that doctors and lawyers owe their clients. Under this paradigm, conversational agents would be legally required to act in the user's best interest, not merely optimize for engagement or platform goals. The paper advocates for embedding this principle into the very design of the interface and user experience, making fiduciary obligations visible and enforceable. If adopted, this could fundamentally reshape how AI companies design and deploy conversational tools, especially in regulated domains like healthcare and finance.
- Proposes that conversational AI be legally classified as 'digital fiduciaries' with a duty of care.
- Argues that current alignment research overlooks the need for a normative legal baseline.
- Aims to unify trust and accountability into a single design and legal paradigm, presented at CUI '26.
Why It Matters
Could force AI companies to redesign assistants for regulated domains with legal accountability, not just technical alignment.