OpenIdDict using PKCS#11 / HSM interface?

Question

I am working on a project that uses OpenIdDict für user authentication, thus, generating and validating JWTs (Java web tokens, HTTP header "Bearer").

Currently, the prototype relies on a certificate file residing in the file system, but eventually, it should use real HSM infrastructure, either accessible via a PKCS#11 library, or indirectly via OpenSSL (or similar). Target system is Linux.

The question is whether and how OpenIdDict can be configured to use PKCS#11 instead of a X509 certificate, for token generation and validation?

Currently, the web server (which only needs to verify JWTs from incoming messages) is configured as:

using Microsoft.IdentityModel.Protocols.OpenIdConnect;
using Microsoft.IdentityModel.Tokens;
using Microsoft.OpenApi.Models;
using OpenIddict.Validation.AspNetCore;
using System.Security.Cryptography.X509Certificates;
[...]
services.AddOpenIddict()
            .AddValidation(options =>
[...]
                var issuerSigningKey = new X509Certificate2(Path, Password);

[...]
                var oidcConfig = new OpenIdConnectConfiguration
                {
                    // Issuer ...
                };
                oidcConfig.SigningKeys.Add(new X509SecurityKey(issuerSigningKey));
                options.SetConfiguration(oidcConfig);

Whereas the authentication server (which creates the JWTs) is configured with:

return builder.AddServer(options =>
    {
...
        options.AddSigningCertificate(new X509Certificate2(Path, Password));                           

There seems, at least for the web server, the possibility to inject a custom JWT validator delegate to the underlying JwtSecurityTokenHandler.cs, but that means that it needs to implement all the JWT's detail checks on its own. I would prefer to rely on the library's infrastructure and only configure which HSM token to address, somehow.