sendnrw/misskey
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf(federation): Ed25519署名に対応する (#13464)

Browse Source 
* 1. ed25519キーペアを発行・Personとして公開鍵を送受信

* validate additionalPublicKeys

* getAuthUserFromApIdはmainを選ぶ

* ✌️

* fix

* signatureAlgorithm

* set publicKeyCache lifetime

* refresh

* httpMessageSignatureAcceptable

* ED25519_SIGNED_ALGORITHM

* ED25519_PUBLIC_KEY_SIGNATURE_ALGORITHM

* remove sign additionalPublicKeys signature requirements

* httpMessageSignaturesSupported

* httpMessageSignaturesImplementationLevel

* httpMessageSignaturesImplementationLevel: '01'

* perf(federation): Use hint for getAuthUserFromApId (#13470)

* Hint for getAuthUserFromApId

* とどのつまりこれでいいのか？

* use @misskey-dev/node-http-message-signatures

* fix

* signedPost, signedGet

* ap-request.tsを復活させる

* remove digest prerender

* fix test?

* fix test

* add httpMessageSignaturesImplementationLevel to FederationInstance

* ManyToOne

* fetchPersonWithRenewal

* exactKey

* ✌️

* use const

* use gen-key-pair fn. from  '@misskey-dev/node-http-message-signatures'

* update node-http-message-signatures

* fix

* @misskey-dev/node-http-message-signatures@0.0.0-alpha.11

* getAuthUserFromApIdでupdatePersonの頻度を増やす

* cacheRaw.date

* use requiredInputs
https://github.com/misskey-dev/misskey/pull/13464#discussion_r1509964359

* update @misskey-dev/node-http-message-signatures

* clean up

* err msg

* fix(backend): fetchInstanceMetadataのLockが永遠に解除されない問題を修正

Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>

* fix httpMessageSignaturesImplementationLevel validation

* fix test

* fix

* comment

* comment

* improve test

* fix

* use Promise.all in genRSAAndEd25519KeyPair

* refreshAndprepareEd25519KeyPair

* refreshAndfindKey

* commetn

* refactor public keys add

* digestプリレンダを復活させる

RFC実装時にどうするか考える

* fix, async

* fix

* !== true

* use save

* Deliver update person when new key generated (not tested)
https://github.com/misskey-dev/misskey/pull/13464#issuecomment-1977049061

* 循環参照で落ちるのを解消？

* fix?

* Revert "fix?"

This reverts commit 0082f6f8e8.

* a

* logger

* log

* change logger

* 秘密鍵の変更は、フラグではなく鍵を引き回すようにする

* addAllKnowingSharedInboxRecipe

* nanka meccha kaeta

* delivre

* キャッシュ有効チェックはロック取得前に行う

* @misskey-dev/node-http-message-signatures@0.0.3

* PrivateKeyPem

* getLocalUserPrivateKey

* fix test

* if

* fix ap-request

* update node-http-message-signatures

* fix type error

* update package

* fix type

* update package

* retry no key

* @misskey-dev/node-http-message-signatures@0.0.8

* fix type error

* log keyid

* logger

* db-resolver

* JSON.stringify

* HTTP Signatureがなかったり使えなかったりしそうな場合にLD Signatureを活用するように

* inbox-delayed use actor if no signature

* ユーザーとキーの同一性チェックはhostの一致にする

* log signature parse err

* save array

* とりあえずtryで囲っておく

* fetchPersonWithRenewalでエラーが起きたら古いデータを返す

* use transactionalEntityManager

* fix spdx

* @misskey-dev/node-http-message-signatures@0.0.10

* add comment

* fix

* publicKeyに配列が入ってもいいようにする
https://github.com/misskey-dev/misskey/pull/13950

* define additionalPublicKeys

* fix

* merge fix

* refreshAndprepareEd25519KeyPair → refreshAndPrepareEd25519KeyPair

* remove gen-key-pair.ts

* defaultMaxListeners = 512

* Revert "defaultMaxListeners = 512"

This reverts commit f2c412c180.

* genRSAAndEd25519KeyPairではキーを直列に生成する?

* maxConcurrency: 8

* maxConcurrency: 16

* maxConcurrency: 8

* Revert "genRSAAndEd25519KeyPairではキーを直列に生成する?"

This reverts commit d0aada55c1.

* maxWorkers: '90%'

* Revert "maxWorkers: '90%'"

This reverts commit 9e0a93f110.

* e2e/timelines.tsで個々のテストに対するtimeoutを削除, maxConcurrency: 32

* better error handling of this.userPublickeysRepository.delete

* better comment

* set result to keypairEntityCache

* deliverJobConcurrency: 16, deliverJobPerSec: 1024, inboxJobConcurrency: 4

* inboxJobPerSec: 64

* delete request.headers['host'];

* fix

* // node-fetch will generate this for us. if we keep 'Host', it won't change with redirects!

* move delete host

* modify comment

* modify comment

* fix correct → collect

* refreshAndfindKey → refreshAndFindKey

* modify comment

* modify attachLdSignature

* getApId, InboxProcessorService

* TODO

* [skip ci] add CHANGELOG

---------

Co-authored-by: MeiMei <30769358+mei23@users.noreply.github.com>
Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>
This commit is contained in:
tamaina
2024-07-18 01:28:17 +09:00
committed by GitHub
parent 3331f3972a
commit 5f88d56d96
52 changed files with 1096 additions and 694 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
packages/backend/test/e2e/timelines.ts
View File

@@ -378,7 +378,7 @@ describe('Timelines', () => {
assert.strictEqual(res.body.some(note => note.id === bobNote2.id), true);
assert.strictEqual(res.body.some(note => note.id === carolNote1.id), false);
assert.strictEqual(res.body.some(note => note.id === carolNote2.id), false);
}, 1000 * 10);
});
test.concurrent('フォローしているユーザーのチャンネル投稿が含まれない', async () => {
const [alice, bob] = await Promise.all([signup(), signup()]);
@@ -672,7 +672,7 @@ describe('Timelines', () => {
assert.strictEqual(res.body.some(note => note.id === bobNote1.id), false);
assert.strictEqual(res.body.some(note => note.id === bobNote2.id), true);
}, 1000 * 10);
});
});
describe('Social TL', () => {
@@ -812,7 +812,7 @@ describe('Timelines', () => {
assert.strictEqual(res.body.some(note => note.id === bobNote1.id), false);
assert.strictEqual(res.body.some(note => note.id === bobNote2.id), true);
}, 1000 * 10);
});
});
describe('User List TL', () => {
@@ -1025,7 +1025,7 @@ describe('Timelines', () => {
assert.strictEqual(res.body.some(note => note.id === bobNote1.id), false);
assert.strictEqual(res.body.some(note => note.id === bobNote2.id), true);
}, 1000 * 10);
});
test.concurrent('リスインしているユーザーの自身宛ての visibility: specified なノートが含まれる', async () => {
const [alice, bob] = await Promise.all([signup(), signup()]);
@@ -1184,7 +1184,7 @@ describe('Timelines', () => {
assert.strictEqual(res.body.some(note => note.id === bobNote1.id), false);
assert.strictEqual(res.body.some(note => note.id === bobNote2.id), true);
}, 1000 * 10);
});
test.concurrent('[withChannelNotes: true] チャンネル投稿が含まれる', async () => {
const [alice, bob] = await Promise.all([signup(), signup()]);

2
packages/backend/test/misc/mock-resolver.ts
View File

@@ -14,6 +14,7 @@ import type { InstanceActorService } from '@/core/InstanceActorService.js';
import type { LoggerService } from '@/core/LoggerService.js';
import type { MetaService } from '@/core/MetaService.js';
import type { UtilityService } from '@/core/UtilityService.js';
import type { FederatedInstanceService } from '@/core/FederatedInstanceService.js';
import { bindThis } from '@/decorators.js';
import type {
FollowRequestsRepository,
@@ -47,6 +48,7 @@ export class MockResolver extends Resolver {
{} as HttpRequestService,
{} as ApRendererService,
{} as ApDbResolverService,
{} as FederatedInstanceService,
loggerService,
);
}

23
packages/backend/test/unit/FetchInstanceMetadataService.ts
View File

@@ -75,62 +75,61 @@ describe('FetchInstanceMetadataService', () => {
test('Lock and update', async () => {
redisClient.set = mockRedis();
const now = Date.now();
federatedInstanceService.fetch.mockResolvedValue({ infoUpdatedAt: { getTime: () => { return now - 10 * 1000 * 60 * 60 * 24; } } } as any);
federatedInstanceService.fetch.mockResolvedValue({ infoUpdatedAt: new Date(now - 10 * 1000 * 60 * 60 * 24) } as any);
httpRequestService.getJson.mockImplementation(() => { throw Error(); });
const tryLockSpy = jest.spyOn(fetchInstanceMetadataService, 'tryLock');
const unlockSpy = jest.spyOn(fetchInstanceMetadataService, 'unlock');
await fetchInstanceMetadataService.fetchInstanceMetadata({ host: 'example.com' } as any);
expect(federatedInstanceService.fetch).toHaveBeenCalledTimes(1);
expect(tryLockSpy).toHaveBeenCalledTimes(1);
expect(unlockSpy).toHaveBeenCalledTimes(1);
expect(federatedInstanceService.fetch).toHaveBeenCalledTimes(1);
expect(httpRequestService.getJson).toHaveBeenCalled();
});
test('Lock and don\'t update', async () => {
test('Don\'t lock and update if recently updated', async () => {
redisClient.set = mockRedis();
const now = Date.now();
federatedInstanceService.fetch.mockResolvedValue({ infoUpdatedAt: { getTime: () => now } } as any);
federatedInstanceService.fetch.mockResolvedValue({ infoUpdatedAt: new Date() } as any);
httpRequestService.getJson.mockImplementation(() => { throw Error(); });
const tryLockSpy = jest.spyOn(fetchInstanceMetadataService, 'tryLock');
const unlockSpy = jest.spyOn(fetchInstanceMetadataService, 'unlock');
await fetchInstanceMetadataService.fetchInstanceMetadata({ host: 'example.com' } as any);
expect(tryLockSpy).toHaveBeenCalledTimes(1);
expect(unlockSpy).toHaveBeenCalledTimes(1);
expect(federatedInstanceService.fetch).toHaveBeenCalledTimes(1);
expect(tryLockSpy).toHaveBeenCalledTimes(0);
expect(unlockSpy).toHaveBeenCalledTimes(0);
expect(httpRequestService.getJson).toHaveBeenCalledTimes(0);
});
test('Do nothing when lock not acquired', async () => {
redisClient.set = mockRedis();
const now = Date.now();
federatedInstanceService.fetch.mockResolvedValue({ infoUpdatedAt: { getTime: () => now - 10 * 1000 * 60 * 60 * 24 } } as any);
federatedInstanceService.fetch.mockResolvedValue({ infoUpdatedAt: new Date(now - 10 * 1000 * 60 * 60 * 24) } as any);
httpRequestService.getJson.mockImplementation(() => { throw Error(); });
await fetchInstanceMetadataService.tryLock('example.com');
const tryLockSpy = jest.spyOn(fetchInstanceMetadataService, 'tryLock');
const unlockSpy = jest.spyOn(fetchInstanceMetadataService, 'unlock');
await fetchInstanceMetadataService.fetchInstanceMetadata({ host: 'example.com' } as any);
expect(federatedInstanceService.fetch).toHaveBeenCalledTimes(1);
expect(tryLockSpy).toHaveBeenCalledTimes(1);
expect(unlockSpy).toHaveBeenCalledTimes(0);
expect(federatedInstanceService.fetch).toHaveBeenCalledTimes(0);
expect(httpRequestService.getJson).toHaveBeenCalledTimes(0);
});
test('Do when lock not acquired but forced', async () => {
test('Do when forced', async () => {
redisClient.set = mockRedis();
const now = Date.now();
federatedInstanceService.fetch.mockResolvedValue({ infoUpdatedAt: { getTime: () => now - 10 * 1000 * 60 * 60 * 24 } } as any);
federatedInstanceService.fetch.mockResolvedValue({ infoUpdatedAt: new Date(now - 10 * 1000 * 60 * 60 * 24) } as any);
httpRequestService.getJson.mockImplementation(() => { throw Error(); });
await fetchInstanceMetadataService.tryLock('example.com');
const tryLockSpy = jest.spyOn(fetchInstanceMetadataService, 'tryLock');
const unlockSpy = jest.spyOn(fetchInstanceMetadataService, 'unlock');
await fetchInstanceMetadataService.fetchInstanceMetadata({ host: 'example.com' } as any, true);
expect(federatedInstanceService.fetch).toHaveBeenCalledTimes(0);
expect(tryLockSpy).toHaveBeenCalledTimes(0);
expect(unlockSpy).toHaveBeenCalledTimes(1);
expect(federatedInstanceService.fetch).toHaveBeenCalledTimes(0);
expect(httpRequestService.getJson).toHaveBeenCalled();
});
});

90
packages/backend/test/unit/ap-request.ts
View File

@@ -4,10 +4,8 @@
*/
import * as assert from 'assert';
import httpSignature from '@peertube/http-signature';
import { genRsaKeyPair } from '@/misc/gen-key-pair.js';
import { ApRequestCreator } from '@/core/activitypub/ApRequestService.js';
import { verifyDraftSignature, parseRequestSignature, genEd25519KeyPair, genRsaKeyPair, importPrivateKey } from '@misskey-dev/node-http-message-signatures';
import { createSignedGet, createSignedPost } from '@/core/activitypub/ApRequestService.js';
export const buildParsedSignature = (signingString: string, signature: string, algorithm: string) => {
return {
@@ -24,38 +22,68 @@ export const buildParsedSignature = (signingString: string, signature: string, a
};
};
describe('ap-request', () => {
test('createSignedPost with verify', async () => {
const keypair = await genRsaKeyPair();
const key = { keyId: 'x', 'privateKeyPem': keypair.privateKey };
const url = 'https://example.com/inbox';
const activity = { a: 1 };
const body = JSON.stringify(activity);
const headers = {
'User-Agent': 'UA',
};
async function getKeyPair(level: string) {
if (level === '00') {
return await genRsaKeyPair();
} else if (level === '01') {
return await genEd25519KeyPair();
}
throw new Error('Invalid level');
}
const req = ApRequestCreator.createSignedPost({ key, url, body, additionalHeaders: headers });
describe('ap-request post', () => {
const url = 'https://example.com/inbox';
const activity = { a: 1 };
const body = JSON.stringify(activity);
const headers = {
'User-Agent': 'UA',
};
const parsed = buildParsedSignature(req.signingString, req.signature, 'rsa-sha256');
describe.each(['00', '01'])('createSignedPost with verify', (level) => {
test('pem', async () => {
const keypair = await getKeyPair(level);
const key = { keyId: 'x', 'privateKeyPem': keypair.privateKey };
const result = httpSignature.verifySignature(parsed, keypair.publicKey);
assert.deepStrictEqual(result, true);
});
const req = await createSignedPost({ level, key, url, body, additionalHeaders: headers });
test('createSignedGet with verify', async () => {
const keypair = await genRsaKeyPair();
const key = { keyId: 'x', 'privateKeyPem': keypair.privateKey };
const url = 'https://example.com/outbox';
const headers = {
'User-Agent': 'UA',
};
const parsed = parseRequestSignature(req.request);
expect(parsed.version).toBe('draft');
expect(Array.isArray(parsed.value)).toBe(false);
const verify = await verifyDraftSignature(parsed.value as any, keypair.publicKey);
assert.deepStrictEqual(verify, true);
});
test('imported', async () => {
const keypair = await getKeyPair(level);
const key = { keyId: 'x', 'privateKey': await importPrivateKey(keypair.privateKey) };
const req = ApRequestCreator.createSignedGet({ key, url, additionalHeaders: headers });
const req = await createSignedPost({ level, key, url, body, additionalHeaders: headers });
const parsed = buildParsedSignature(req.signingString, req.signature, 'rsa-sha256');
const result = httpSignature.verifySignature(parsed, keypair.publicKey);
assert.deepStrictEqual(result, true);
const parsed = parseRequestSignature(req.request);
expect(parsed.version).toBe('draft');
expect(Array.isArray(parsed.value)).toBe(false);
const verify = await verifyDraftSignature(parsed.value as any, keypair.publicKey);
assert.deepStrictEqual(verify, true);
});
});
});
describe('ap-request get', () => {
describe.each(['00', '01'])('createSignedGet with verify', (level) => {
test('pass', async () => {
const keypair = await getKeyPair(level);
const key = { keyId: 'x', 'privateKeyPem': keypair.privateKey };
const url = 'https://example.com/outbox';
const headers = {
'User-Agent': 'UA',
};
const req = await createSignedGet({ level, key, url, additionalHeaders: headers });
const parsed = parseRequestSignature(req.request);
expect(parsed.version).toBe('draft');
expect(Array.isArray(parsed.value)).toBe(false);
const verify = await verifyDraftSignature(parsed.value as any, keypair.publicKey);
assert.deepStrictEqual(verify, true);
});
});
});