perf(federation): Ed25519署名に対応する (#13464)

* 1. ed25519キーペアを発行・Personとして公開鍵を送受信

* validate additionalPublicKeys

* getAuthUserFromApIdはmainを選ぶ

* ✌️

* fix

* signatureAlgorithm

* set publicKeyCache lifetime

* refresh

* httpMessageSignatureAcceptable

* ED25519_SIGNED_ALGORITHM

* ED25519_PUBLIC_KEY_SIGNATURE_ALGORITHM

* remove sign additionalPublicKeys signature requirements

* httpMessageSignaturesSupported

* httpMessageSignaturesImplementationLevel

* httpMessageSignaturesImplementationLevel: '01'

* perf(federation): Use hint for getAuthUserFromApId (#13470)

* Hint for getAuthUserFromApId

* とどのつまりこれでいいのか？

* use @misskey-dev/node-http-message-signatures

* fix

* signedPost, signedGet

* ap-request.tsを復活させる

* remove digest prerender

* fix test?

* fix test

* add httpMessageSignaturesImplementationLevel to FederationInstance

* ManyToOne

* fetchPersonWithRenewal

* exactKey

* ✌️

* use const

* use gen-key-pair fn. from  '@misskey-dev/node-http-message-signatures'

* update node-http-message-signatures

* fix

* @misskey-dev/node-http-message-signatures@0.0.0-alpha.11

* getAuthUserFromApIdでupdatePersonの頻度を増やす

* cacheRaw.date

* use requiredInputs
https://github.com/misskey-dev/misskey/pull/13464#discussion_r1509964359

* update @misskey-dev/node-http-message-signatures

* clean up

* err msg

* fix(backend): fetchInstanceMetadataのLockが永遠に解除されない問題を修正

Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>

* fix httpMessageSignaturesImplementationLevel validation

* fix test

* fix

* comment

* comment

* improve test

* fix

* use Promise.all in genRSAAndEd25519KeyPair

* refreshAndprepareEd25519KeyPair

* refreshAndfindKey

* commetn

* refactor public keys add

* digestプリレンダを復活させる

RFC実装時にどうするか考える

* fix, async

* fix

* !== true

* use save

* Deliver update person when new key generated (not tested)
https://github.com/misskey-dev/misskey/pull/13464#issuecomment-1977049061

* 循環参照で落ちるのを解消？

* fix?

* Revert "fix?"

This reverts commit 0082f6f8e8.

* a

* logger

* log

* change logger

* 秘密鍵の変更は、フラグではなく鍵を引き回すようにする

* addAllKnowingSharedInboxRecipe

* nanka meccha kaeta

* delivre

* キャッシュ有効チェックはロック取得前に行う

* @misskey-dev/node-http-message-signatures@0.0.3

* PrivateKeyPem

* getLocalUserPrivateKey

* fix test

* if

* fix ap-request

* update node-http-message-signatures

* fix type error

* update package

* fix type

* update package

* retry no key

* @misskey-dev/node-http-message-signatures@0.0.8

* fix type error

* log keyid

* logger

* db-resolver

* JSON.stringify

* HTTP Signatureがなかったり使えなかったりしそうな場合にLD Signatureを活用するように

* inbox-delayed use actor if no signature

* ユーザーとキーの同一性チェックはhostの一致にする

* log signature parse err

* save array

* とりあえずtryで囲っておく

* fetchPersonWithRenewalでエラーが起きたら古いデータを返す

* use transactionalEntityManager

* fix spdx

* @misskey-dev/node-http-message-signatures@0.0.10

* add comment

* fix

* publicKeyに配列が入ってもいいようにする
https://github.com/misskey-dev/misskey/pull/13950

* define additionalPublicKeys

* fix

* merge fix

* refreshAndprepareEd25519KeyPair → refreshAndPrepareEd25519KeyPair

* remove gen-key-pair.ts

* defaultMaxListeners = 512

* Revert "defaultMaxListeners = 512"

This reverts commit f2c412c180.

* genRSAAndEd25519KeyPairではキーを直列に生成する?

* maxConcurrency: 8

* maxConcurrency: 16

* maxConcurrency: 8

* Revert "genRSAAndEd25519KeyPairではキーを直列に生成する?"

This reverts commit d0aada55c1.

* maxWorkers: '90%'

* Revert "maxWorkers: '90%'"

This reverts commit 9e0a93f110.

* e2e/timelines.tsで個々のテストに対するtimeoutを削除, maxConcurrency: 32

* better error handling of this.userPublickeysRepository.delete

* better comment

* set result to keypairEntityCache

* deliverJobConcurrency: 16, deliverJobPerSec: 1024, inboxJobConcurrency: 4

* inboxJobPerSec: 64

* delete request.headers['host'];

* fix

* // node-fetch will generate this for us. if we keep 'Host', it won't change with redirects!

* move delete host

* modify comment

* modify comment

* fix correct → collect

* refreshAndfindKey → refreshAndFindKey

* modify comment

* modify attachLdSignature

* getApId, InboxProcessorService

* TODO

* [skip ci] add CHANGELOG

---------

Co-authored-by: MeiMei <30769358+mei23@users.noreply.github.com>
Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>

packages/backend/src/server/ActivityPubServerService.ts

@@ -3,11 +3,10 @@
  * SPDX-License-Identifier: AGPL-3.0-only
  */
 
-import * as crypto from 'node:crypto';
 import { IncomingMessage } from 'node:http';
 import { Inject, Injectable } from '@nestjs/common';
 import fastifyAccepts from '@fastify/accepts';
-import httpSignature from '@peertube/http-signature';
+import { verifyDigestHeader, parseRequestSignature } from '@misskey-dev/node-http-message-signatures';
 import { Brackets, In, IsNull, LessThan, Not } from 'typeorm';
 import accepts from 'accepts';
 import vary from 'vary';
@@ -31,12 +30,17 @@ import { IActivity } from '@/core/activitypub/type.js';
 import { isQuote, isRenote } from '@/misc/is-renote.js';
 import type { FastifyInstance, FastifyRequest, FastifyReply, FastifyPluginOptions, FastifyBodyParser } from 'fastify';
 import type { FindOptionsWhere } from 'typeorm';
+import { LoggerService } from '@/core/LoggerService.js';
+import Logger from '@/logger.js';
 
 const ACTIVITY_JSON = 'application/activity+json; charset=utf-8';
 const LD_JSON = 'application/ld+json; profile="https://www.w3.org/ns/activitystreams"; charset=utf-8';
 
 @Injectable()
 export class ActivityPubServerService {
+	private logger: Logger;
+	private inboxLogger: Logger;
+
 	constructor(
 		@Inject(DI.config)
 		private config: Config,
@@ -71,8 +75,11 @@ export class ActivityPubServerService {
 		private queueService: QueueService,
 		private userKeypairService: UserKeypairService,
 		private queryService: QueryService,
+		private loggerService: LoggerService,
 	) {
 		//this.createServer = this.createServer.bind(this);
+		this.logger = this.loggerService.getLogger('server-ap', 'gray');
+		this.inboxLogger = this.logger.createSubLogger('inbox', 'gray');
 	}
 
 	@bindThis
@@ -100,70 +107,44 @@ export class ActivityPubServerService {
 	}
 
 	@bindThis
-	private inbox(request: FastifyRequest, reply: FastifyReply) {
-		let signature;
+	private async inbox(request: FastifyRequest, reply: FastifyReply) {
+		if (request.body == null) {
+			this.inboxLogger.warn('request body is empty');
+			reply.code(400);
+			return;
+		}
+
+		let signature: ReturnType<typeof parseRequestSignature>;
+
+		const verifyDigest = await verifyDigestHeader(request.raw, request.rawBody || '', true);
+		if (verifyDigest !== true) {
+			this.inboxLogger.warn('digest verification failed');
+			reply.code(401);
+			return;
+		}
 
 		try {
-			signature = httpSignature.parseRequest(request.raw, { 'headers': [] });
-		} catch (e) {
+			signature = parseRequestSignature(request.raw, {
+				requiredInputs: {
+					draft: ['(request-target)', 'digest', 'host', 'date'],
+				},
+			});
+		} catch (err) {
+			this.inboxLogger.warn('signature header parsing failed', { err });
+
+			if (typeof request.body === 'object' && 'signature' in request.body) {
+				// LD SignatureがあればOK
+				this.queueService.inbox(request.body as IActivity, null);
+				reply.code(202);
+				return;
+			}
+
+			this.inboxLogger.warn('signature header parsing failed and LD signature not found');
 			reply.code(401);
 			return;
 		}
 
-		if (signature.params.headers.indexOf('host') === -1
-			|| request.headers.host !== this.config.host) {
-			// Host not specified or not match.
-			reply.code(401);
-			return;
-		}
-
-		if (signature.params.headers.indexOf('digest') === -1) {
-			// Digest not found.
-			reply.code(401);
-		} else {
-			const digest = request.headers.digest;
-
-			if (typeof digest !== 'string') {
-				// Huh?
-				reply.code(401);
-				return;
-			}
-
-			const re = /^([a-zA-Z0-9\-]+)=(.+)$/;
-			const match = digest.match(re);
-
-			if (match == null) {
-				// Invalid digest
-				reply.code(401);
-				return;
-			}
-
-			const algo = match[1].toUpperCase();
-			const digestValue = match[2];
-
-			if (algo !== 'SHA-256') {
-				// Unsupported digest algorithm
-				reply.code(401);
-				return;
-			}
-
-			if (request.rawBody == null) {
-				// Bad request
-				reply.code(400);
-				return;
-			}
-
-			const hash = crypto.createHash('sha256').update(request.rawBody).digest('base64');
-
-			if (hash !== digestValue) {
-				// Invalid digest
-				reply.code(401);
-				return;
-			}
-		}
-
 		this.queueService.inbox(request.body as IActivity, signature);
-
 		reply.code(202);
 	}
 
@@ -640,7 +621,7 @@ export class ActivityPubServerService {
 			if (this.userEntityService.isLocalUser(user)) {
 				reply.header('Cache-Control', 'public, max-age=180');
 				this.setResponseType(request, reply);
-				return (this.apRendererService.addContext(this.apRendererService.renderKey(user, keypair)));
+				return (this.apRendererService.addContext(this.apRendererService.renderKey(user, keypair.publicKey)));
 			} else {
 				reply.code(400);
 				return;

packages/backend/src/server/NodeinfoServerService.ts

@@ -94,6 +94,13 @@ export class NodeinfoServerService {
 					localComments: 0,
 				},
 				metadata: {
+					/**
+					 * '00': Draft, RSA only
+					 * '01': Draft, Ed25519 suported
+					 * '11': RFC 9421, Ed25519 supported
+					 */
+					httpMessageSignaturesImplementationLevel: '01',
+
 					nodeName: meta.name,
 					nodeDescription: meta.description,
 					nodeAdmins: [{

packages/backend/src/server/api/endpoints/admin/queue/inbox-delayed.ts

@@ -56,7 +56,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
 			const res = [] as [string, number][];
 
 			for (const job of jobs) {
-				const host = new URL(job.data.signature.keyId).host;
+				const signature = job.data.signature ? 'version' in job.data.signature ? job.data.signature.value : job.data.signature : null;
+				const host = signature ? Array.isArray(signature) ? 'TODO' : new URL(signature.keyId).host : new URL(job.data.activity.actor).host;
 				if (res.find(x => x[0] === host)) {
 					res.find(x => x[0] === host)![1]++;
 				} else {