The Agent Network, Part 2 of 3: Interfaces, Evaluation, and Shared Memory

Generate the interface, then save it

A dropdown and a multi-select can launch an agent workflow. But requests for richer controls can turn a simple platform into an entire interface builder. Agent.ai faces that pressure: its interface collects inputs, runs workflow steps, and returns HTML or markup. Meanwhile, delegation may be asynchronous, like handing a coworker something to finish in an hour or a day. The interface has to accommodate that handoff.

The proposed solution in Part 2 of The Agent Network is to generate an interface for one workflow step, then save its code. A builder describes the interaction, iterates through prompts, and saves the result as that step’s action. The generated code becomes a stable artifact for later execution. This is a planned feature, with predictability as a central motivation.

The analogy is giving a language model a Python interpreter: the model delegates concrete computation to executable code. Interface generation could follow a similar pattern, either generating at runtime or iterating toward a cached result. Another participant adds a boundary: most agents inside a composed workflow may need no interface, because they pass work onward through webhooks. The design question is where a human should intervene.

The participant implementing the feature uses Python and Cursor, and argues that Python is expressive enough while remaining a common language humans can read and work with. Another participant proposes a related middle ground between prebuilt components and whole-app generation: pin a generated form and let research populate it. The form stays stable while research fills its blanks. Further out is the possibility that AI could invent interaction primitives beyond today’s checkboxes, dropdowns, gestures, and voice.

Evaluate agents on the work you need

Moving roles to AI—what one participant calls AI shoring—raises a hiring question. Human assessment often relies on historical signals because direct evaluation time is limited. Agents offer another possibility: run candidates on the task and compare demonstrated work.

Every Agent.ai agent is described as automatically exposing a callable API. That could support automated selection: try several agents against a use case, evaluate their outputs, and choose the best fit. Publishers might allow 50 or 100 trial calls, with an audit trail recording usage. Human reviews offer another signal: the platform reportedly had tens of thousands, averaging roughly 4.1 out of five. But a platform rating and suitability for a particular task answer different questions.

For an AI software engineer, the proposed assessment is concrete: 1. Assemble 15–20 examples of actual organizational issues, covering collaboration and code generation. 2. Pay to run candidates through them, much as an organization might pay people for take-home projects. 3. Evaluate the resulting work against the organization’s needs. The obstacle is making evaluations—often shortened to evals—routine. Participants acknowledge underinvesting in them themselves; one relies on personal or colleague review while recognizing the need for a scalable approach.

Choose models per step—and earn trust in the choice

A question about competition with OpenAI brings the discussion back to Agent.ai’s origins. It grew from solo software: personal productivity projects that increasingly used language models and needed a common development framework. Its builder expects OpenAI to keep investing in an agent ecosystem after plugins, custom GPTs, and the GPT Store, explicitly making a prediction without inside information.

The rationale for an independent framework is model choice at each workflow step. A writing step might use Claude; image generation might use different models depending on whether the image contains text. Provider neutrality lets builders mix, replace, and test those choices for each use case.

Users reportedly tend to choose the highest version number, including GPT-4.5, even when automatic selection appears first in the menu. The builder attributes this partly to distrust of the automatic choice and reports substantial inference bills. The commitment at the time is to support all models anyway, treating Agent.ai as a research effort and hoping falling inference costs make that sustainable.

An experiment under discussion would repeatedly run the same agent—1,000 runs is the example—and use ongoing human ratings to assess model substitutions, starting with internal agents. The builder claims that suitable tasks can move to less capable models for savings of multiple orders of magnitude without reducing output quality, while also avoiding extra latency. The savings claim is task-dependent and unbenchmarked here: the 1,000 runs are an experimental example, with no named model comparison, exact costs, or formal quality measure.

Runtime model routing means selecting a model for a task as the workflow executes. Asked whether to buy or build it, the builder strongly prefers buying an existing capability. Another participant describes Chai AI as a marketplace where model providers compete alongside the other participants. The economic goal is to match the right model to the right use case at the right price; discovery, trust, and pricing can each prevent that match.

DSPy enters the exchange as a possible framework for this work. With acknowledged limited familiarity, participants describe its appeal as establishing evaluations first, then using them to optimize model choices.

Outcome pricing needs measurable, comparable results

The business-model discussion separates three things a customer might buy: - Software as a service: a human uses software to perform work. - Work as a service: software performs a task, such as classification or contract review. - Results as a service: payment depends on the outcome produced. Automating a task does not by itself determine how to price it.

One participant argues that enthusiasm for outcome pricing draws too heavily from customer support. Per-resolved-ticket pricing has a plausible basis there: teams already know roughly what resolution costs and have a semi-objective service standard. If, for example, the existing threshold is customers being happy with 90% of handled tickets, the AI must meet that same standard.

The proposed test has two dimensions: - Can the outcome be measured objectively? Buyer and provider need a workable definition of success. - Does value per outcome vary little? A common unit price is harder to justify when outcomes differ enormously in economic value. Logo design exposes the first problem. Satisfaction might take five iterations or 100, depending partly on the customer’s taste and ability to articulate it. Fixed fees with bounded revision rounds are one way design services contain that uncertainty.

A podcast-logo commission through 99designs makes the ambiguity tangible: designers worked hard, but the buyer did not know what they wanted. Another participant recommends guaranteeing that a winner will be paid to attract more participation. More candidates can improve the chance of finding a suitable design, while also creating more noise. The buyer’s uncertainty remains part of the work.

Two further objections concern the economics of the final result: - Buyers may want to retain the upside. Paying for work lets a customer keep the difference between its cost and the value they generate from it. - A provider’s contribution may be hard to isolate. When many inputs produce an outcome, attributing it to one service becomes difficult.

The attribution problem prompts a prediction that some underlying Web3 ideas will return: trackable digital assets, fractional ownership, and publicly verifiable audit logs. The argument is limited to use cases where those benefits justify blockchain storage overhead. Another barrier is incentives: organizations holding the data needed for attribution may have little reason to share it.

More engineering capacity, more solvable problems

Applied to engineering, the response is optimistic. More human and digital engineers increase the supply of labor, but AI tools may also expand the economic value of problems engineers can solve. One participant’s thesis is that newly accessible value could grow faster than engineering supply, raising engineers’ long-term value.

Several compensation models could coexist: work for hire, payment for a delivered artifact, and agents sold through a marketplace. The practical advice is to learn by using and building agents, including outside engineering roles. For engineers, another participant points toward memory and planning as areas where the agent stack remains immature and open to substantial development.

Shared memory needs selective access

Cross-agent memory lets something one agent learns about a user become useful context for another. Agent.ai’s builder reports working on this capability: with user opt-in, people should not need to repeat the same information to every agent. Shared context becomes a reason to build on an agent network. The desired approach is privacy-oriented, with trust and verifiability central, rather than a promise that privacy always overrides every other consideration.

Scheduling turns that ambition into an authorization problem. Another person should be able to discover available times and see shared events without learning about unrelated calendar entries. Useful access can be narrower than account access. OAuth 2.0 is described as the closest existing authorization mechanism, but its provider-defined scopes—the permissions users approve—can be coarse. Providers could define finer scopes; the builder expects that change to be slow.

Email supplies a more consequential example. One participant uses it as a personal event and data bus, gathering information there because its open protocol keeps that information accessible for later use. Yet the same concentration of personal information makes unrestricted startup access unacceptable: too much of their life is in the inbox.

The proposed architecture inserts a trusted intermediary that enforces narrower sharing policies. It would obtain underlying access through OAuth, then accept controls expressed in natural language, such as: - Only specified labels: restrict which messages a downstream provider receives. - No more than 50 emails per day: limit disclosure volume so the provider cannot download the full backlog. The proposal concentrates trust in the intermediary: it needs broad underlying access to enforce narrow access downstream. Adoption depends on users trusting it enough to grant those permissions.

The memory survey names MemGPT, now Letta, and LangMem from LangGraph as relevant projects. One participant also points toward background memory processing, likening it to AI sleep. A separate extension concerns who shares the memory: beyond one user’s agents, a team could have private shared state across its people and agents, analogous to a Slack channel.

Bee is mentioned as working on selective conversation sharing, followed by a tentative reference to voice-based shielding at Limitless. The exchange includes investment disclosures: one participant reports investments in Limitless, OpenAI, Perplexity, LangGraph, and Crew AI, while disclaiming inside knowledge; another describes being a friend of Bee rather than an investor.

Domains can carry ideas—and open investment doors

The conversation shifts to projects that may never get built. One participant describes buying domains after late-night ideas and reports owning Scout.ai and Agent.com, with the latter tied to Agent.ai. A possible handoff is to give a domain to someone whose one-page proposal shows a plausible idea and willingness to execute. This remains a possibility, not an announced program.

The Chat.com backstory begins with a mistaken expectation. Its buyer initially thought ChatGPT would demonstrate OpenAI’s platform, leaving room for a consumer product with a simpler name. The buyer acquired the domain with that product in mind, then interpreted ChatGPT’s plugins as evidence that the application itself was becoming a platform. That reinforced a personal rule against competing with Sam Altman.

Other parties had been interested in the domain, but the buyer says their identities were unknown. After acquiring it, the buyer approached Sam, offered it without seeking a profit, and reached a deal, reasoning that OpenAI could do something bigger with it. A subsequent discussion of Agent.ai’s domain evaluator adds a limitation: the published V1 did not use transaction data. A version incorporating that data remained unpublished and was described as operationally intensive.

The buyer also reports owning crew.ai and offering to sell it at cost when the intended recipient is ready. Asked for buying advice, the buyer describes negotiating personally under their own identity. Credibility can improve response rates, even if recognition raises asking prices; receiving a higher asking price does not require accepting it.

playground.com illustrates another use for domain ownership. The original idea was a provider-neutral interface for trying multiple language models, inspired by OpenAI’s Playground. Its owner ultimately took equity in a deal involving Suhail’s Playground company, describing domains as a way into investments otherwise difficult to access.

Market efficiency returns as a transaction problem: domain purchases can involve escrow and transfers taking days or weeks. A participant reports investing in an unnamed company aiming to make domains fractionalizable and tradable. The desired experience is pay, then control the domain within minutes. Another participant compares it to Ethereum Name Service; the response emphasizes bringing that ease of use to ordinary buyers and conventional domains.

Conviction over a long horizon

Asked what sustains personal drive, one participant credits conviction and persistence: stay with an idea until convinced it is wrong, or until technology makes it feasible. A roughly 20-year interest in natural language is offered as an example of waiting for the world to catch up. Competition is framed through chess: study the game over time and treat competitors as opponents rather than enemies.

That horizon becomes concrete at HubSpot: the speaker says they have been there for 18 years and intend to remain another 18, while emphasizing that the market leaves room for others. Part 2 ends with an unanswered question: how do you develop conviction when you begin with little knowledge, and keep going when setbacks erode it?