Research Statement
_(6197849910).jpg){kind=link}
This research statement may be continuously updated as projects develop and change how I approach the broader questions.
Overview
How can we deliberately engineer agents and organized systems of agents? When can a collection of interacting components be treated as a single agent? How do organizations arise and persist “naturally”, and how do their forms depend on incentives, the environment, and the demands placed on them? How do agents develop purposes, methods of communication, and standards of judgment, and how can an organization of agents maintain and revise shared purposes over time? Can we develop engineering measures, analogous to factors of safety in civil engineering, that quantify how much disturbance or uncertainty these systems can tolerate while still behaving as intended?
My current research, mostly shared on this blog (when completed), attempts to answer these questions abstractly and mathematically, and to build tools and infrastructure for better managing multi-agent systems in their real world. Below, I outline my strategy for approaching these and related questions. The program is organized around specific subquestions and resulting projects, each of which contributes towards investigating the broader goals.
Agent Foundations
This part of the program concerns the definition, origin, and control of agents. Before we can deliberately produce or control agency, we need criteria for recognizing it and an account of the processes that generate it. Those foundations should tell us which properties we can engineer and under what conditions.1
What Are Agents, and How Can We Recognize Them?
In order to engineer the properties of agents, we first need to define what an agent is, and then we need reliable ways to recognize an agent in the context of a broader system. What distinguishes agency from passive persistence? Does agency depend on the observer of system definition? What physical processes support an agent boundary? How does the answer depend on the timescale, spatial scale, or the behavior we desire to explain? Tests for agency would let us compare candidate definitions and determine which systems call for an explanation in terms of agents.
- The Making of an Agent [WIP] is the planned synthesis of work on selection, thermodynamics, and distinction-cost (see below). How can locally maintained structures acquire information, alter their environments, and repair or reproduce the mechanisms that keep them viable, without goals being specified in advance? It includes the questions of how such structures acquire boundaries and when their joint maintenance justifies treating them as one agent. The aim is to identify conditions under which agency can be produced reliably.
How Do Agents Form, and How Can We Create Them Reliably?
If we want to produce agents, we should understand where they come from in the first place. How can agency arise without agents or purposes being specified in advance?
Primitive Selection
Primitive selection is differential persistence among structures or trajectories, without assuming agents, goals, or biological reproduction. Can it favor structures that acquire information, respond to disturbances, and maintain the mechanisms responsible for those responses? I want to identify when this provides a route from passive structures to increasingly capable agents.
- Inspection Bias works through the established result that longer-lived objects are overrepresented in observations taken at a point in time. It establishes how persistence changes the distribution we observe, providing the baseline effect that later projects build on to explain apparent purpose.
- Selection (starter post) [WIP] is a planned exposition bringing inspection bias and selection as conditioning into one account. How does filtering a population change its observed distribution, and when does repeated selection produce a directional change? It supplies the starting point for the agency argument.
- Fixation Ratchets [WIP] asks whether information that improves persistence can become a shared baseline, allowing subsequent selection to act on further differences. The proposed mechanism connects information advantages to cumulative development: what conditions let useful information accumulate across successive rounds, and when do changing environments, loss, or insufficient variation interrupt the process?
Information, Control, and Physical Maintenance
Can acquiring and using information improve persistence enough to pay for the mechanisms involved? When do survival-conditioned dynamics correspond to a control problem, and what would it take for local physical processes to implement that control? This is needed to establish when information and control confer a sustainable advantage, and when their costs prevent an agent from forming.
- Thermodynamics from a Systems Perspective [WIP] is a planned exposition of the conditions under which a dynamical system admits a thermodynamic description. What assumptions about coarse-graining, initial conditions, and dynamics are needed for entropy growth? This supplies the physical premises needed to investigate the maintenance and formation of agents.
- Power and Distinction: How Many Symbols Can Be Maintained [WIP] is a planned account of the resources required to maintain reliable distinctions. How do resolution, error tolerance, and the physical implementation determine that cost? It develops a common foundation for agent formation, symbolic coordination, and category systems, separating the cost of sustaining distinctions from the uses to which they are put.
- Emergence from Persistence: Inspection Bias as a Generative Principle [WIP] is a working draft extending persistence weighting to combinations of controllers and the structures they sustain. Can a composite persist when its isolated parts cannot, and when does that justify treating it as a new unit? It connects the agency argument to coalition formation and the maintenance of symbols.
How Can We Control Agents, and What Are the Limits of Such Control?
Once we can recognize and produce agents, how can we influence what they do? Which aspects of their behavior can be controlled through available interventions, and which depend on internal processes or environmental conditions we cannot reliably observe or change? These limits determine which behaviors we can deliberately produce and what reliability we can expect.
What limits follow from incomplete observations, finite communication, and the time and resources available for computation? How do these limits change when the system being controlled can learn, adapt, or deliberately respond to the controller? The aim is to determine when better control requires more information, different capabilities, or a different organization.
Geometric Control
Geometric control describes how available actions move a system through its possible states. I use this viewpoint to relate the controls we can apply to the trajectories and maintained properties we want.
- Controls from the Geometric Perspective develops the mathematical background for relating control problems to variational and Hamiltonian descriptions. Starting from available actions and desired trajectories, it investigates how to formulate the corresponding optimization problem, supplying tools for designing and analyzing control.
- Noether’s Theorem and Geometric Controls connects symmetries of variational systems to conserved quantities. I use it to ask which maintained properties of agents admit such a description, and under what assumptions conservation laws could contribute to their analysis and control.
Organization
To engineer systems of agents, we need to understand both what the members do and what their interaction makes possible. When is a collective description justified, and what determines the organization’s form and ability to act on shared purposes?
When Can a Collection of Agents Be Treated as a Single Agent?
The boundary problem recurs when several agents are grouped together. When can the group be represented as one agent while preserving its relevant behavior and capacity for control? What internal coordination must sustain that description, and how do its informational and physical costs affect which coalitions form and persist? A reliable collective description would let us predict and control the group without tracking every member separately.
- Game Decompositions [WIP] is a working manuscript on breaking games into simpler interactions. The planned starter brings together subgames, Galois-style symmetry reduction, and resolvent descriptions of local dynamics. I want to determine which parts of a multi-agent problem can be analyzed separately, and quantify the errors introduced when their separation is only approximate.
- Compressed Controllers and Coalition Compression [WIP] asks when a coalition can be coordinated through substantially fewer variables than a description of all its members would require. It connects game decomposition to the agency argument: when does a compressed representation remain useful under interventions, and can the savings in coordination help explain why a larger agent forms and persists?
How Does Organizational Form Depend on Scale, Environment, and Demands?
An organization that works under one set of conditions may cease to work when it grows or its task changes. Which features of its form follow from the demands placed on it? When can an existing arrangement be extended, and when must its members coordinate differently? The aim is to explain observed forms and identify arrangements suited to a specified task and environment.
Form and Scaling
This work studies how the relationships among an organization’s parts change with its size, workload, or available resources. Understanding those dependencies should help explain why particular forms work at particular scales, and when growth requires a different arrangement.
Dynamical Similarity and Equivariant Symmetry examines transformations that relate dynamical systems across changes of scale. This supplies a way to ask which behaviors can remain similar as a system grows, and which changes require a different organization.
Algebra and Allometry uses invariant theory to organize scaling relationships, including cases where several quantities change together. The intent is to understand how assumptions about scaling constrain possible forms, and when growth requires a change in proportion or structure. One proposed empirical application, How Many Elites Does It Take to Run a System? [WIP], asks how the size of a decision-making layer scales with the system it governs, and which demands or constraints explain differences between organizations.
Symmetry-Structured Theory of Organizations [WIP] is a working draft using symmetry groups and representations to constrain possible organizational roles, relations, and forms. Which of these possibilities arise under given resources and dynamics, and how long do they persist? Can reconstruction after damage help distinguish an organization from a temporarily persistent arrangement? The aim is to connect structural constraints to the formation and maintenance of organizations, producing predictions that can be checked against observed systems.
Institutional Universality Classes [WIP] is a proposed classification of organizations by the effective strategic structures that survive aggregation of their internal detail. The initial target is systems with a principal and many agents. If different organizations fall into the same class, this could let us transfer predictions about coordination and failure between them, with explicit conditions on what the classification preserves.
Coordination, Power, and Institutional Change
This work examines how coordination arrangements distribute decision-making power and dependence among members. I want to understand how technological or institutional changes alter that distribution, and what makes collective control durable or vulnerable.
- Thoughts on Selectorate Theory uses a model of ruling coalitions to investigate how dependence on supporters shapes resource allocation and political stability. It supplies a concrete case for the link between organizational structure and incentives. Under linearity and budget invariance, its hierarchy calculation also reduces a subordinate level to the effective cost share needed for the level above to secure loyalty.
- Does AI Make Totalitarianism More Likely? examines how changes in the costs of information, coordination, and coercion could alter the balance between centralized and distributed power. It applies the organizational questions to a concrete technological change: which dependencies keep leaders responsive to other people, and what happens when those dependencies weaken?
How Does the Structure of an Interaction Shape Outcomes?
The same agents can cooperate in one setting and work against one another in another. Which properties of their interaction account for the difference? To change the outcome deliberately, we need to understand both the incentives they face and how their behavior develops over time.
Classifying Strategic Interactions
Classification groups interactions according to specified features of their incentives and strategic dependencies. This would let us recognize when apparently different situations pose the same coordination problem, and identify which changes actually alter that problem.
- Differentiable Game Canonicalization builds a differentiable classifier for strict ordinal two-player, two-strategy games. This was my first computational approach to identifying an interaction’s strategic type from its payoff table, with the further aim of making that classification usable inside learning and design procedures.
- Invariant Coordinates for Normal-Form Games Modulo Strategy Relabeling develops an algebraic representation of games that is unchanged by strategy relabeling. The paper computes explicit coordinates in small cases and investigates larger games, providing a more systematic basis for comparing interactions and tracking how their structure changes. The companion Games, Invariants, and Alignment explains how this classification could guide changes in incentives that support cooperation.
Learning, Cooperation, and Strategic Dynamics
This work studies how behavior develops as agents learn and respond to one another. To design a cooperative arrangement, we need to understand whether agents can reach a cooperative outcome, sustain it, and recover it after a disturbance.
- Differential Games and Stag Hunt constructs a version of stag hunt with continuous states and actions, alongside a framework for simulating differential games. I use it to investigate cooperation as a process sustained through time, providing a setting for experiments in multi-agent control and collective behavior.
How Can We Engineer Reliable Organizations?
Understanding why an organization behaves as it does should help us design one that behaves as intended. Which changes are available to a designer, and how much variation in incentives, membership, or environment can an arrangement tolerate before it no longer serves its purpose? I want bounds on where an organizational design remains dependable, how it can fail, and what margins would justify relying on it.
Designing Incentives and Organizational Structure
This concerns choosing how tasks, information, rewards, and decision-making authority are distributed among agents. The aim is to make desired collective behavior achievable through the actions and judgments of the members, while accounting for the costs of coordination.
- Delegation as Information Bottleneck: The Compression Limits of Principal-Agent Problems [WIP] is a proposed account of how instructions, incentives, selection, and monitoring communicate what a principal wants. How do these channels trade off against one another, and which errors can better communication resolve? Which failures remain because the agents have conflicting interests? The aim is to connect organizational design to the practical limits of specifying and evaluating delegated work.
- Charting Game Paths [WIP] develops computational tools for controlling game type by changing incentives. The implemented examples trace transitions between strategic regimes as design parameters vary; the planned account connects those paths to feasible interventions, asking who can change which parameters and whether the resulting dynamics support cooperation.
- Inverse Organizational Design [WIP] is a proposed program for working from requirements to organizational structure. Given a task, resources, costs, and expected disturbances, can we generate and compare candidate roles, relations, and repair mechanisms? The aim is to turn the organizational theory into a design procedure that explains the tradeoffs between candidate forms. A longer-term goal is to accompany recommendations with explicit, model-dependent bounds on performance and failure.
Value Formation
This part of the program studies how agents develop purposes and standards of judgment through experience and interaction. Understanding that process is necessary if we want to engineer agents that can evaluate unfamiliar possibilities and organizations that can maintain and revise shared purposes.
How Do Agents Learn What to Value?
Encountering something new can change what an agent wants and the distinctions it uses to judge. How can we model learning when the standards of evaluation are themselves developing? What makes a change in judgment informed or useful, and how could an agent learn to recognize that? The aim is to build learning procedures that can improve judgment as new possibilities become available.
Specification and Evaluation
Specification concerns expressing what we want; evaluation concerns judging what we receive or encounter. Understanding the limits of both is necessary for deciding what can be communicated in advance and what an agent must learn through interaction and feedback.
- The Paradox of Taste: Borges, Preference Oracles, and the Platonic Theory of Art asks what it would take to find a work we would value among an enormous space of possible works. The essay develops the problem of taste as a problem of recognizing and selecting what is worth producing, even when generation is available.
- Is a Picture Worth a Thousand Words? examines how much information is needed to specify an intended image, and what a short prompt leaves unresolved. It connects the difficulty of expressing a preference to the practical need for shared context, interpretation, and feedback when delegating creative work.
Learning Through Experience and Other Agents
This work investigates how exposure, criticism, and relationships change evaluative judgment. The aim is to develop ways for agents to learn from other agents whose knowledge and standards differ, including how to decide whose judgments deserve trust.
- Towards Social Learning [WIP] is a working draft asking whether agents can learn to evaluate without having their rewards specified in advance. It develops the connection between experience, changing representations, criticism, and learning whose judgment to trust, with the aim of making the formation of evaluative standards part of the learning process. One proposed mechanism, Predicted Regret and the Social Counterfactual Sampler, uses the experiences of comparable agents who chose differently as evidence about choices an agent did not make. When is that comparison informative, given differences between agents and the selection of experiences they observe? Can anticipating how it might later judge those alternatives help an agent revise its current standards?
- Judging Social Learning [WIP] is a proposed experimental program separating three questions: did social information change an agent’s evaluator, what patterns of agreement and disagreement developed in the population, and did the resulting evaluators help on later problems? Studying stability, diversity, and transitions between collective states would help characterize the learning process. Varying communication, reputation, and authority would also test how organizational arrangements shape value formation.
Can We Align Agents to Human Value Production?
I want to investigate whether we can align AI with the processes through which humans develop values. People revise their judgments as they encounter new possibilities, so alignment needs an account of how an agent should respond to those changes. Which experiences, criticisms, and relationships should influence its judgment, and whose judgment should it trust? How can it support people in revising their purposes while preserving their authority to decide which changes to accept? AI systems also influence the experiences and judgments from which they learn. How can we evaluate that coupled process, including whether it expands people’s ability to reflect, disagree, and revise their purposes?
- Alignment and Ethics proposes a consistency test for ethical judgments: what should change when we regroup the same underlying system into different agents or collectives? It also considers the unproven possibility of an objective whose preferred actions agree across those levels. This anchors the value-formation work in the question of whose interests count and which constraints should apply as purposes change.
How Does the Organization of Society Affect What Becomes Valuable?
What people can come to value also depends on what they encounter, who can participate, and which commitments their institutions sustain. How do changes in production, communication, and social organization alter those possibilities? How does something valued by individuals become something a society recognizes and supports? I want to understand which social arrangements enable valuable activities to develop and persist, particularly as the conditions of production change.
- Are We Approaching Cultural Saturation? asks why recognizable cultural forms recur despite the enormous number of possible works. It explores how abstraction, limited attention, and the effort required to distinguish works could constrain novelty. The aim is to understand what limits cultural development and whether expanding the supply of works changes those limits.
- Financial Explanations of Art examines how auctions, museums, collectors, and other institutions confer financial value on art. The aim is to understand how expectations about other people’s recognition and commitment connect shared cultural judgments to prices.
- Markets for Symbols [WIP] is a working draft on how maintained distinctions acquire coordination value. Building on Power and Distinction, it asks how attention, recognition, and sustained commitments constrain the symbols a community can support, and how their value is expressed through markets and institutions. The aim is to connect cultural value formation to measurable constraints on sustaining collective recognition. A related question is when a symbol’s coordination value supports the continued viability of those who use it, and when the two diverge.
- Thoughts on Demand argues that greater productive capacity need not translate directly into proportionate economic growth. It investigates the discovery, trust, purchasing power, and coordination needed to turn wants into demand, connecting technological abundance to the institutions through which people decide what is worth supporting.
- How Will Humans Generate Value in a Post-AI Society? takes cheap, highly automated production as a premise and asks which activities would still matter to people. It explores how participation, relationships, status, and meaning could shape value when the effort required to produce an object becomes less central.
Methods and Infrastructure
The questions above require ways to recover structure from data, compare alternative models, and check what follows from their assumptions. This work develops the mathematical and computational tools I use for those tasks, alongside research software that makes the ideas possible to explore.
How much of a system’s organization can we infer from its observed behavior? Which alternative explanations fit the same observations, and what additional measurements or interventions would distinguish them? These questions matter whenever we infer an agent, a control process, or a collective from the behavior of its parts.
How Can We Systematically Construct, Compare, and Check Theories?
Given assumptions about a system’s components and transformations, which models are compatible with them? Can we enumerate those possibilities within useful limits, determine which are equivalent, and compute where their predictions differ? I want these tools to support both explanations of observed systems and the search for systems with desired properties. I build small implementations to understand the methods, test their assumptions, and expose steps that can be automated.
- Building a Minimal Computational Invariant Theory Library implements algorithms for constructing and checking invariants. The aim is to make structural classification something I can compute and reuse across different mathematical and physical systems. The accompanying Survey of Classical Invariant Theory develops the mathematical background used in these constructions and their applications to games and physical models.
- From Symmetry to Theory: A Computational Engine implements an engine for constructing symmetry-compatible expressions within specified choices of fields, transformations, and complexity bounds. The longer-term aim is a searchable catalogue of admissible models and their consequences, making it easier to compare theories and identify candidates for a desired behavior.
- Symmetry Engine II: A Platform for Approximate Symmetry [WIP] is the second Autophysics post, extending the engine to broken, approximate, and observationally indistinguishable symmetries. Its examples compute newly permitted terms, drifting quantities, and properties that survive perturbation. The aim is to make model construction and comparison useful for systems whose symmetries are imperfect or change.
Why It Matters
The aim is to understand agents and organizations well enough to engineer them deliberately. As more work and decision-making are delegated to AI, we need ways to build reliable systems, adapt the institutions around them, and make use of the growing volume of information they produce.
AI Alignment and Multi-Agent Systems
AI alignment requires us to build systems that do what we intend and to determine when we can rely on them. This includes whole systems of models, tools, operators, and institutions. We need engineering measures of how much uncertainty, disturbance, or conflict these systems can tolerate while still behaving as intended. Alignment also depends on how their purposes are formed and revised. I want to investigate how AI can be aligned with human processes of learning, criticism, and value development, while preserving people’s authority over the purposes these systems serve.
Institutions and Governance
I want to use this work to build better governments, corporations, and other institutions: systems that coordinate effectively, remain accountable to the people they serve, and can correct failures or change direction. As AI changes the costs of production, information, and coordination, we need ways to adapt these organizations and decide how work, resources, and authority should be distributed. A theory of organizations should help us compare possible designs and anticipate their effects on cooperation, concentrations of power, and people’s ability to shape collective decisions.
Biology
Living systems exhibit agency and organization across several scales: cells, organisms, colonies, and ecosystems. The framework may help distinguish genuine higher-level control from a convenient description. Biology also supplies cases in which boundaries, coordination, and responses to disturbances must arise and be maintained without an external designer specifying the resulting agent.
Science and Engineering
The growing volume of publications makes it difficult to know what has already been established, how results relate, and which claims deserve further attention. AI-assisted research may make this problem more acute. We need better ways to organize and compress scientific information while preserving the distinctions needed to understand and use it. Structural representations, searchable catalogs, and machine-checkable derivations could help us recognize equivalent results, keep assumptions attached to claims, and find theories or methods suited to a problem. The aim is to make accumulated knowledge easier to navigate and build on, and to make the search for systems with desired properties more systematic.
Footnotes
Note that this program mostly remains agnostic to “consciousness” and concerns itself with only agency.↩︎