Missing domain error

[tk-o]

Transaction details: https://etherscan.io/tx/0x109e7530e5c1d5d4e45c2623df553901d470117a31704e5793efd3152b5c89b4

9:02:22 AM WARN user Failed 'domains.update()' database method (id=edc0065d)

RecordNotFoundError: No existing record found in table 'domains'

Full error message

9:02:19 AM INFO  app        Indexed 3055 events with 93.4% complete and 20s remaining

9:02:22 AM WARN  user       Failed 'domains.update()' database method (id=edc0065d)

RecordNotFoundError: No existing record found in table 'domains'

    at file:///app/node_modules/.pnpm/ponder@0.8.28_@opentelemetry+api@1.9.0_@types+node@20.17.10_@types+pg@8.11.10_hono@4.6.14_typ_k2ne6a4izfbuk3jfnqcbtlq2cy/node_modules/ponder/src/indexing-store/historical.ts:658:35

    at processTicksAndRejections (node:internal/process/task_queues:105:5)

    at HeadlessKysely.wrap (file:///app/node_modules/.pnpm/ponder@0.8.28_@opentelemetry+api@1.9.0_@types+node@20.17.10_@types+pg@8.11.10_hono@4.6.14_typ_k2ne6a4izfbuk3jfnqcbtlq2cy/node_modules/ponder/src/database/kysely.ts:58:24)

    at handleNameTransferred (/app/src/handlers/Registrar.ts:151:7)

    at /app/src/plugins/eth/handlers/EthRegistrar.ts:18:5

    at executeEvent (file:///app/node_modules/.pnpm/ponder@0.8.28_@opentelemetry+api@1.9.0_@types+node@20.17.10_@types+pg@8.11.10_hono@4.6.14_typ_k2ne6a4izfbuk3jfnqcbtlq2cy/node_modules/ponder/src/indexing/service.ts:440:5)

    at processEvents (file:///app/node_modules/.pnpm/ponder@0.8.28_@opentelemetry+api@1.9.0_@types+node@20.17.10_@types+pg@8.11.10_hono@4.6.14_typ_k2ne6a4izfbuk3jfnqcbtlq2cy/node_modules/ponder/src/indexing/service.ts:255:20)

    at handleEvents (file:///app/node_modules/.pnpm/ponder@0.8.28_@opentelemetry+api@1.9.0_@types+node@20.17.10_@types+pg@8.11.10_hono@4.6.14_typ_k2ne6a4izfbuk3jfnqcbtlq2cy/node_modules/ponder/src/bin/utils/run.ts:78:12)

    at start2 (file:///app/node_modules/.pnpm/ponder@0.8.28_@opentelemetry+api@1.9.0_@types+node@20.17.10_@types+pg@8.11.10_hono@4.6.14_typ_k2ne6a4izfbuk3jfnqcbtlq2cy/node_modules/ponder/src/bin/utils/run.ts:166:22)

db.update arguments:

  id   0x1f8a792bf0baa71294e95daffc24c70673ba2b5ea993305bc9f6da6f05cca594

9:02:22 AM ERROR indexing   Error while processing '/eth/BaseRegistrar:Transfer' event in 'mainnet' block 20334227

RecordNotFoundError: No existing record found in table 'domains'

    at handleNameTransferred (/app/src/handlers/Registrar.ts:151:7)

    at /app/src/plugins/eth/handlers/EthRegistrar.ts:18:5

  149 |       await context.db.update(schema.registration, { id: label }).set({ registrantId: to });

  150 |

> 151 |       await context.db.update(schema.domain, { id: node }).set({ registrantId: to });

      |       ^

  152 |

  153 |       // TODO: log Event

  154 |     },

db.update arguments:

  id   0x1f8a792bf0baa71294e95daffc24c70673ba2b5ea993305bc9f6da6f05cca594

Event arguments:

  from      0x0000000000000000000000000000000000000000

  to        0x283Af0B28c62C092C9727F1Ee09c02CA627EB7F5

  tokenId   58401712555322395348808557205735895619909073721831328983501574527606232333348

Log:

  index     343

  address   0x57f1887a8BF19b14fC0dF6Fd9B2acc9Af147eA85

Transaction:

  hash   0x109e7530e5c1d5d4e45c2623df553901d470117a31704e5793efd3152b5c89b4

  from   0xf540fdFA0a08303C918ad383251ce8C48c42C39A

  to     0x283Af0B28c62C092C9727F1Ee09c02CA627EB7F5

Block:

  hash        0x030557525855c23cc183f507bde245227646c8ddc1811e1864e3392e34132a52

  number      20334227

  timestamp   1721315603

9:02:22 AM INFO  app        Indexed 3403 events with 93.4% complete and 21s remaining

9:02:22 AM WARN  process    Encountered indexing error, starting shutdown sequence

9:02:22 AM FATAL process    Finished shutdown sequence, terminating (exit code 1)

 ELIFECYCLE  Command failed with exit code 1.

[tk-o]

Looking at [the related transaction logs], the very first event in the log is causing the runtime error on the eth plugin. Looking into the handler code, it looks like the registration entity check would pass, and then the domain update action would throw the runtime error:

ensnode/src/handlers/Registrar.ts

Lines 146 to 151 in 64054e1

	const registration = await context.db.find(schema.registration, { id: label });
	if (!registration) return;
	await context.db.update(schema.registration, { id: label }).set({ registrantId: to });
	await context.db.update(schema.domain, { id: node }).set({ registrantId: to });

Debugging context

Labelhash & Namehash

A registration entity uses the labelhash value as its primary key.
A domain entity uses the namehash value as its primary key.

I've checked the labelhash and namehash values at https://tools.ens.xyz/check/quantamm.eth
Hamehash: 0x1f8a792bf0baa71294e95daffc24c70673ba2b5ea993305bc9f6da6f05cca594
Labelhash: 0x811e32aa3574546fc4571994c19c547f5b37329c4bc749494ef33e446a363c24

Event log

1. The error occurs while processing the very first event on the transaction.

2. The logs would suggest that the registration entity should be inserted into database while handling the third event in that transaction

3. The error can only happen, if the registration entity for the aforementioned labelhash was inserted prior handling events from the 0x109e7530e5c1d5d4e45c2623df553901d470117a31704e5793efd3152b5c89b4 transaction.

What do you think about that @shrugs & @lightwalker-eth?

[tk-o]

Now I think we might have a cross-chain registration conflict. The quantamm.linea.eth domain was registered before the quantamm.eth one.

We have the same label registered in two different parent domains:

• https://app.ens.domains/quantamm.linea.eth
  • registration date: Jul-12-2024 10:23:36 AM UTC
  • tx link: https://lineascan.build/tx/0x6dbe18798f9918444843f39815d1676959bae8a65e48eb2ae942df5da0e1031a
• https://app.ens.domains/quantamm.eth
  • registration date: Jul-18-2024 03:13:23 PM UTC
  • tx link: https://etherscan.io/tx/0x109e7530e5c1d5d4e45c2623df553901d470117a31704e5793efd3152b5c89b4

[lightwalker-eth]

@tk-o Fantastic debugging!

I believe this issue might be related to the goals I was hoping to advance in #44

We need to separate out the following ideas:

• NFT Token Id
• Labelhash
• Namehash

In the BaseRegistrarController for .eth names, the labelhash of the direct subname of .eth is used as the NFT token id.

In the NameWrapper (for all ENS names), the namehash (of the entire name) is used as the NFT token id.

The labelhash of a subname is not safe for use as a unique identifier across all of ENS. Only the namehash of the entire name is.

[lightwalker-eth]

Please also note how the ENS Protocol says nothing at all about NFTs or NFT token ids. The implementation details of NFTs and NFT token ids are separate ideas from ENS itself.

[shrugs]

aha! so because Registration.id is keyed by labelhash (since the subgraph only expected to track registrations under .eth) that logic passes but there's no existing domain entity for the second node that's being transferred. We definitely can't change the key of Registration within the eth plugin, as that would affect subgraph compatibility.

perhaps a simple solution, though it isn't very pretty, is to just pass another argument to makeRegistrationHandlers(keyPrefix = '') and pass that plugin's name or whatever for base and linea. then update every handler to construct a registration id like:

const registrationId = `${keyPrefix}${label}`

but we should add a comment about this, and note in v2.md that we want to correctly key these things in the future (likely by namehash). wdyt?

[lightwalker-eth]

@shrugs Could you please verify that the subgraph only tracks registrations under .eth and not at all within the NameWrapper (which goes beyond .eth, and also uses a different token id strategy for .eth NFTs).

Could you please help me better understand the background / goals of what we need a function like makeRegistrationHandlers to do? Is the goal to make a universally unique id for a specific registration event, or for all registration events associated with the same name? Or something else?

[tk-o]

I've checked the stats for the www label with these queries:

-- labelhash(`www`)=`0x3177317affd6342ed2401ccea23053d41b86d0914b5b1bee0faa1efcb7221a61`

SELECT * FROM "25ad4bb7-8697-4788-bf11-5253ca96026c".registrations
WHERE "id" = '0x3177317affd6342ed2401ccea23053d41b86d0914b5b1bee0faa1efcb7221a61';

SELECT * FROM "25ad4bb7-8697-4788-bf11-5253ca96026c".domains 
WHERE labelhash = '0x3177317affd6342ed2401ccea23053d41b86d0914b5b1bee0faa1efcb7221a61'; 

There's just one record in registration table, and 216 records in the domains table.

[tk-o]

This issue exists on the Subgraph API as well: check out this query.

In query results, only one domain record links to its registration record, while all others just show registration: null.

[shrugs]

I mean that because the Registrations entity is keyed by labelhash, the subgraph encoded an assumption that a Registration is specifically a .eth subdomain registration, and yes a Registration record is only handled in the context of a Registrar in this case the .eth one. It never imagined creating Registration records for any other registrar. a subgraph 'Registration' has no relation to anything happening in the NameWrapper

wdym different tokenId strategy? ETH Registrar and NameWrapper both use labelhash encoded as unit256, right?

the makeRegistrationHandlers function is a factory function to provide additional injected context to the otherwise generic handlers that we're providing to ponder for those events. in this case we already need to inject the ownedName variable, because a Registrar's handlers all operate within the context of a specific ownedName (like .eth or .base.eth).

what tko has discovered is that there is another registrar-specific piece of context that these otherwise-generic handlers need, and it's to further scope the Registration record's id value, since the original code (which derived from the subgraph) encoded the assumption mentioned above. the correct solution is to key Registratioins by namehash, but that would affect subgraph compatibility for the .eth plugin, because then the Registration record's id would be different. This is not something we can do at the moment, so providing a keyPrefix argument to the factory function allows the generic handler code to stay logically equivalent in the .eth plugin but provide a unique id for the Registrations created by the new base and linea plugins (which don't have the same subgraph-compatibility constraint)

[shrugs]

In query results, only one domain record links to its registration record, while all others just show registration: null.

yes, which is by design. the Registration record only tracks .eth Registrar events, and the others are subdomains of those domains. only one of these domains is a direct subdomain of .eth (it's www.eth) and that's the one that has the Registration record. the other domains were not 'Registered' by a 'Registrar' according to the subgraph, since the subgraph only tracks ETHRegistry events.

[shrugs]

the .eth plugin, when run alone, will never run into this issue because labelhash is a perfectly good unique id for a Registration record (again because the assumption is that a Registration is specifically representing an action taken in the ETHRegistrar), as long as one does not index any other Registrars. we have discovered this issue because that's exactly what we're doing.

like mentioned, the solution is a complete refactor of the schema, to better represent the actual on-chain state and specifics of the ENS protocol. the subgraph encodes many assumptions, and this is one of them.

[lightwalker-eth]

Thanks guys. Several points:

1. Within the context of the ENS Subgraph it makes sense that the NameWrapper doesn't emit any Registration records itself and that only the .eth BaseRegistrar does. The NameWrapper and BaseRegistrar are both "Registrar Controllers" for .eth, so I thought for a moment that they both might emit Registration records, but the NameWrapper is implemented such that it makes calls into the BaseRegistrar for .eth registrations. Therefore all Registration records are emitted by the BaseRegistrar. Makes sense!
   1. NOTE: The NameWrapper can wrap (essentially) any ENS name, not just direct subnames of .eth. But the NameWrapper knows nothing about "registrations" of all these names. There's only a single contract in the ENS Subgraph that knows anything about registrations, and that's the BaseRegistrar.
2. @tk-o Within the context of the ENS Subgraph, please take special note of the distinction between a domain (any full ENS name) and a registration (which is assumed to always be a registration associated with a direct subname of .eth in the BaseRegistrar).
3. @tk-o Please also note the distinction between a labelhash and a tokenId. For the BaseRegistrar the tokenid of a direct subname is the labelhash of the direct subname. But other than that specific example, labelhash and tokenId are totally distinct concepts. There can be any number of names that share the label "www" (somewhere in their name) and each of those labels will have the same labelhash. Therefore there can be any number of labels with the same labelhash.

wdym different tokenId strategy? ETH Registrar and NameWrapper both use labelhash encoded as unit256, right?

@shrugs I mean that different contracts can use different strategies for deciding on the relationship between a name and a tokenId. The BaseRegistrar uses the labelhash of the direct subname. The NameWrapper uses the namehash of the full name. Etc.

---

And thanks for the details about makeRegistrationHandlers.

For Subgraph compatibility, it seems we should try to continue using exactly the same Registration id strategy within the .eth BaseRegistrar. However, for our other plugins, couldn't we just do something simple such as prefix those registrations by a fixed constant such as "namehash:{namehash}" or something like that? In other words, we can take advantage of how the .eth BaseRegistrar strategy for Registration ids is just labelhash of the direct subname's label. Therefore, as I understand, in our other plugins, we just need to define a different id issuance strategy that is guaranteed not to conflict.

---

@shrugs Agreed that we want to refactor our schema a lot for V2+, but for V1 as I understand we should continue to implement creative solutions as needed to achieve our multi-plugin goals while still conforming to the ENS Subgraph schema. If your understanding is different please let me know. Cheers.

[tk-o]

Thanks for the extra context, @shrugs. Perhaps, to retain subgraph compatibility, but solve the cross-chain labelhash issue, we could add new subname helpers:

export function ownedNameToTokenIdPrefix(ownedName: `${string}eth`): string {
  return ownedName.endsWith("eth") ? ownedName.slice(-3) : "";
}

export function prefixedTokenIdToLabel(tokenId: bigint, prefix: string): `0x${string}` {
  const tokenIdHex = toHex(tokenId, { size: 32 });

  if (!prefix) {
    return tokenIdHex;
  }

  return concat([toHex(prefix, { size: 32 }), tokenIdHex]);
}

and then use the available ownedName param inside makeRegistrarHandlers factory function to create a prefix-aware version of tokenIdToLabel helper:

const tokenIdToLabel = (tokenId: bigint) => 
  prefixedTokenIdToLabel(tokenId, ownedNameToTokenIdPrefix(ownedName));

[shrugs]

The BaseRegistrar uses the labelhash of the direct subname. The NameWrapper uses the namehash of the full name. Etc.

ah, i see, thanks for the specificity. will include this note in #44

---

correct, a different id issuance strategy is needed for the non-.eth plugins. i proposed a very simple one, with a dumb prefix to the labelhash, as a stopgap for our v1. the namehash: prefix is a very similar idea, yes?

---

@tk-o too complicated, in my opinion. since all we need to do is scope Registration ids by the Registrar that is creating them, i really do like the simple prefix. in fact, we don't need to pass a new parameter at all, we already have ownedName! so the fix looks like

// in handlers/Registrar.ts

// scope non-.eth plugin Registration records by ownedName, to avoid labelhash uniqueness conflicts
// TODO: in v2, key Registrations by namehash
const makeRegistrationId(ownedName: string, labelhash: Hex) => ownedName === '.eth'?  labelhash : `${ownedName}:${labelhash}`;

and then use that helper to construct ever Registration id in that file.

[shrugs]

not sure where you're seeing the need for a tokenIdToLabel update in the context of this issue. the registrars will continue to issue uint256 of labelhash. and we then prefix it for the record's id. don't see a situation in which we need to unpack a prefixed id

[tk-o]

Just went with my gut feeling, sir 🙃 I guess I wanted to avoid refactoring by just replacing the tokenIdToLabel implementation. Your option is way simpler and I like it too. Just replaced .eth with eth:

const makeRegistrationId(ownedName: string, labelhash: Hex) => ownedName === 'eth'?  labelhash : `${ownedName}:${labelhash}`;

[lightwalker-eth]

@shrugs I like your idea for makeRegistrationId. Except perhaps with even more comments with all the relevant background info.