Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Attributes #[MapRequestBody] and #[MapQueryString] to map Request input to typed objects #49138

Open
wants to merge 1 commit into
base: 6.3
Choose a base branch
from

Conversation

Koc
Copy link
Contributor

@Koc Koc commented Jan 27, 2023

Q A
Branch? 6.3
Bug fix? no
New feature? yes
Deprecations? no
Tickets #36037, #36093, #45628, #47425, #49002, #49134
License MIT
Doc PR TBD

Yet another variation of how we can map raw Request data to typed objects with validation. We can even build OpenApi Specification based on this DTO classes using NelmioApiDocBundle.

Usage Example 🔧

#[MapRequestBody]

class PostProductReviewPayload
{
    public function __construct(
        #[Assert\NotBlank]
        #[Assert\Length(min: 10, max: 500)]
        public readonly string $comment,
        #[Assert\GreaterThanOrEqual(1)]
        #[Assert\LessThanOrEqual(5)]
        public readonly int $rating,
    ) {
    }
}

class PostJsonApiController
{
    public function __invoke(
        #[MapRequestBody] PostProductReviewPayload $payload,
    ): Response {
        // $payload is validated and fully typed representation of the request body $request->getContent() 
        //  or $request->request->all()
    }
}

#[MapQueryString]

class GetOrdersQuery
{
    public function __construct(
        #[Assert\Valid]
        public readonly ?GetOrdersFilter $filter,
        #[Assert\LessThanOrEqual(500)]
        public readonly int $limit = 25,
        #[Assert\LessThanOrEqual(10_000)]
        public readonly int $offset = 0,
    ) {
    }
}

class GetOrdersFilter
{
    public function __construct(
        #[Assert\Choice(['placed', 'shipped', 'delivered'])]
        public readonly ?string $status,
        public readonly ?float $total,
    ) {
    }
}

class GetApiController
{
    public function __invoke(
        #[MapQueryString] GetOrdersQuery $query,
    ): Response {
        // $query is validated and fully typed representation of the query string $request->query->all()
    }
}

Exception handling 💥

  • Validation errors will result in an HTTP 422 response, accompanied by a serialized ConstraintViolationList.
  • Malformed data will be responded to with an HTTP 400 error.
  • Unsupported deserialization formats will be responded to with an HTTP 415 error.

Comparison to another implementations 📑

Differences to #49002:

  • separate Attributes for explicit definition of the used source
  • no need to define which class use to map because Attributes already associated with typed argument
  • used ArgumentValueResolver mechanism instead of Subscribers
  • functional tests

Differences to #49134:

  • it is possible to map whole query string to object, not per parameter
  • there is validation of the mapped object
  • supports both $request->getContent() and $request->request->all() mapping
  • functional tests

Differences to #45628:

  • separate Attributes for explicit definition of the used source
  • supports $request->request->all() and $request->query->all() mapping
  • Exception handling opt-in
  • functional tests

Bonus part 🎁

  • Extracted UnsupportedFormatException which thrown when there is no decoder for a given format
  • Added generic Exception Handlers for:
    • PartialDenormalizationException which will convert it to ValidationFailedException
    • ValidationFailedException which will provide HTTP_UNPROCESSABLE_ENTITY Response with serialized to json Constraint Violations

@carsonbot carsonbot added this to the 6.3 milestone Jan 27, 2023
@Koc Koc force-pushed the feature/mapped-request branch 5 times, most recently from 87eb66c to a08be6b Compare January 27, 2023 21:29
@Koc Koc changed the title Create Attributes to map Query String and Request Content to typed objects [HttpKernel] Create Attributes #[MapQueryString and #[MapRequestContent] to map Request input to typed objects Jan 27, 2023
@Koc Koc changed the title [HttpKernel] Create Attributes #[MapQueryString and #[MapRequestContent] to map Request input to typed objects [HttpKernel] Create Attributes #[MapQueryString] and #[MapRequestContent] to map Request input to typed objects Jan 27, 2023
@Koc Koc force-pushed the feature/mapped-request branch 4 times, most recently from 078781e to b3bbe43 Compare January 27, 2023 22:08
@OskarStark
Copy link
Contributor

It looks like your committer email is not associated with your Github account

@ruudk
Copy link
Contributor

ruudk commented Jan 28, 2023

Was this a coincidence that we both did something similar or did you create this afterwards?

@Koc
Copy link
Contributor Author

Koc commented Jan 28, 2023

@OskarStark thanx, fixed now, please check

@ruudk it is similar but has few differences comparing to your implementation. I've updated PR description to highlight the differences.

BTW anybody know how to fix unrelated fabbot failure?

@ro0NL
Copy link
Contributor

ro0NL commented Jan 28, 2023

this looks promising :)

there is validation of the mapped object

what about deserializing errors vs. ValidationFailedException? Does either produce a 400 bad request? is only the latter i18n friendly?

@Koc
Copy link
Contributor Author

Koc commented Jan 28, 2023

My idea was start from something like simple implementation in this PR and improve it later before 6.3 release.

  1. Add generic Exception Handler which will catch ValidationFailedException and convert it to http 400 with serialized Constraint Violations. We already have Normalizer for it but anyway it requires some effort.
  2. Handle PartialDenormalizationException in this two Argument Value Resolvers and convert it to ValidationFailedException. We need somehow translate messages about missmatched types

@ro0NL
Copy link
Contributor

ro0NL commented Jan 28, 2023

i like a simple implementation as first iteration, but i like http/i18n compatibility more ;)

@Koc Koc force-pushed the feature/mapped-request branch 3 times, most recently from 19069b1 to 052f625 Compare January 28, 2023 21:35
@y4roc
Copy link

y4roc commented Jan 30, 2023

Your PR also makes this one #47425 obsolete.

@y4roc
Copy link

y4roc commented Jan 30, 2023

I would use a separate exception, which builds on the ValidationFailedException. So that your listener does not filter out other ValidationFailedException.

@Koc Koc force-pushed the feature/mapped-request branch 2 times, most recently from cf31004 to a2b5f96 Compare January 30, 2023 11:29
@Koc Koc requested review from nicolas-grekas and removed request for dunglas March 4, 2023 10:42
Copy link
Contributor

@soyuka soyuka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the feature that is a shortcut to Deserializing and Validating your data (query or request) before it hits the Controller.
I think that this is helpful but I know it is already possible using API Platform, although API Platform enforces standards (for example for error validation, resource-oriented designs, but also plain json). I'm afraid that these attributes will lead to bad API designs in the future and wanted to share my concern.
Last but not least, the feature you propose looks out of the scope of the HTTP Kernel (adds a dependency on the Serializer and the Validator).

KernelEvents::EXCEPTION => ['onKernelException', -32],
];
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need new exception listeners? Isn't that supposed to be the responsibility of the validation component?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RequestPayloadValueResolver calls Validator, check for result and throws ValidationFailedException for an validation errors. Then we catch this Exception using Listeners mechanism to normalze Violations and respond with proper http code.

@faizanakram99
Copy link
Contributor

faizanakram99 commented Mar 6, 2023

I understand the feature that is a shortcut to Deserializing and Validating your data (query or request) before it hits the Controller. I think that this is helpful but I know it is already possible using API Platform, although API Platform enforces standards (for example for error validation, resource-oriented designs, but also plain json). I'm afraid that these attributes will lead to bad API designs in the future and wanted to share my concern. Last but not least, the feature you propose looks out of the scope of the HTTP Kernel (adds a dependency on the Serializer and the Validator).

We're already using such a similar arg value resolver in house, and from the reaction to this PR (and related PRs or issues), looks like it is not uncommon. It increases overall DX. I shouldn't need to import a big library just to avoid manually mapping json to PHP objects, the framework should do it for me.

@soyuka
Copy link
Contributor

soyuka commented Mar 6, 2023

My argument is more that the join of validation and serialization is probably out of the scope of the HTTP Kernel. Actually what you do here is definitely already possible using the Symfony Framework by doing (pseudo code):

class MyController extends AbstractController {
    #[Route('/route', name: 'app_route')]
    public function myRoute(Request $request): Response
    {
      $serializer = $this->container->get('serializer);
      $data = $serializer->deserialize($request->getContent(), Dto::class, 'json');
      $validator = $this->container->get('validator);
      $validator->validate($data, ...);
    }
}

By "need to import a big library " you're probably talking about API Platform, I understand that this is the current "image" of this framework although it's not that "big" and its integration with the Symfony Framework is quite good.

I see some quite interesting things in the DX here don't get me wrong, I just have some concerns about it being a part of the http-kernel.

@lyrixx
Copy link
Member

lyrixx commented Mar 6, 2023

My argument is more that the join of validation and serialization is probably out of the scope of the HTTP Kernel. Actually what you do here is definitely already possible using the Symfony Framework by doing (pseudo code):

Yes, indeed. Attribute does not allow to do things that are not possible with plain PHP. But it's a shortcut to reduce boilerplate code. Like IsGranted, Cache, MapEntity etc.

So I disagree with you. I'm a heavy user of APIP, but sometime I don't use it and this kind of entity would be very useful!

So it's a big 👍🏼 for me

@faizanakram99
Copy link
Contributor

My argument is more that the join of validation and serialization is probably out of the scope of the HTTP Kernel. Actually what you do here is definitely already possible using the Symfony Framework by doing (pseudo code):

class MyController extends AbstractController {
    #[Route('/route', name: 'app_route')]
    public function myRoute(Request $request): Response
    {
      $serializer = $this->container->get('serializer);
      $data = $serializer->deserialize($request->getContent(), Dto::class, 'json');
      $validator = $this->container->get('validator);
      $validator->validate($data, ...);
    }
}

Indeed, it is possible with a bit of boilerplate, that is specifically what we (users) would like to avoid.

By "need to import a big library " you're probably talking about API Platform, I understand that this is the current "image" of this framework although it's not that "big" and its integration with the Symfony Framework is quite good.

It is big imho, I mean it does a lot more than just deserialize + validate.

I see some quite interesting things in the DX here don't get me wrong, I just have some concerns about it being a part of the http-kernel.

Yeah, the concern about this being part of http-kernel is legit. I do think in an another PR @derrabus had suggested to move the changes to serializer component with optional validation

#45628 (comment)

@soyuka
Copy link
Contributor

soyuka commented Mar 6, 2023

I also saw #49134, I find the implementation quite good and it doesn't bring the issues I see here with the serializer/validator link. I love the idea behind these and this is going to inspire us for some of the upcoming features in API P:D.

It is big imho, I mean it does a lot more than just deserialize + validate.

Thing is it doesn't need to, it can do just the above with about no overhead:

#[Get(uriTemplate: '/books/{id}', controller: MyController::class)]
class Dto {
  public $id;
  #[Assert]
  public $name;
}

class MyController {
  public function __invoke(Dto $data): DtoOutput {
    return new DtoOutput();
  }
}

(NB: pseudo code not sure it works just like that!)

And you can already disable the validation ^^. We're in the process of spliting the code so that one could use only the relevant parts.

/edit: I think we'll find something even better to avoid the Resource approach, I think I get the why you did this, and I quite like the approach!

@Koc Koc force-pushed the feature/mapped-request branch 2 times, most recently from a15fa9d to 25744fd Compare March 6, 2023 16:02
@Koc
Copy link
Contributor Author

Koc commented Mar 10, 2023

@nicolas-grekas rebased again 😃

@ro0NL
Copy link
Contributor

ro0NL commented Mar 14, 2023

btw, given #49614 we should consider #[MapRequestPayload] here (or a recall to consider getBody() there)


$response = new Response(
$data,
Response::HTTP_UNPROCESSABLE_ENTITY,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's reasonable to allow for a custom status code from eg. ValidationFailedException::getStatusCode()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but ValidationFailedException throwed from RequestPayloadValueResolver, you can't throw your own special exception from it.

@Koc Koc requested a review from nicolas-grekas March 15, 2023 13:39
@lyrixx
Copy link
Member

lyrixx commented Mar 15, 2023

Can we merge this one? I have a local branch with some changes (merge of type error + validation ; Drop of partial violation normalizer and other nipticks).

@Koc
Copy link
Contributor Author

Koc commented Mar 15, 2023

Yes, please 🙏. We will have 2 months for tuning it

@hantsy
Copy link

hantsy commented Mar 16, 2023

obsolete

Conversion and validation are two topics.

I am familiar with a few frameworks in Java/Jakarta EE.

Firstly compare to the existing web framework, I would like rename the attributes/annotations in this PR to #[RequestBody], #[RequestForm], and we can also add #[RequestParam], (#[MatrixParam] or request param can parse the matrix params), #[RequestPath], #[RequestPart], #[Header], #[Cookie], #[Session], etc.

<?php

class CreatingUserForm
{
    public function __construct(
        #[RequestParam(name:"name")]
        public readonly string $name,
        #[RequestParam(name:"age", required:false, defaultValue:"0")]
        public readonly int $age= 25,
        #[RequestPart(name:"file")]
        public readonly byte[] $offset = 0,
    ) {
    }
}


class CreatingUserCommand
{
    public function __construct(
        public readonly string $name,
        public readonly int $age= 25
    ) {
    }
}


#[Controller]
class UserApiController
{
// accept a traditional multiform/data
    public function saveFromWeb(
        #[ReuqestForm] CreatingUserForm $body,
    ): Response {
    }


// accept json/xml
 public function saveFromJson(
        #[ReuqestBody] CreatingUserCommand $body,
    ): Response {
    }

}

Through these attrbutes/annotaions to convert to the target object.

Applying the validation is targeted the converted object.

class CreatingUserCommand
{
    public function __construct(

        #[NotBlank]
        public readonly string $name,
        #[Positive]
        public readonly int $age= 25
    ) {
    }
}

// in controller

public function saveFromJson(
        #[ReuqestBody] #[Validated] CreatingUserCommand $body,
    ): Response {
    }

The above dummy usage of attributes/annotations is similar to Jakarta Bean Validations in Spring framework.

@ro0NL
Copy link
Contributor

ro0NL commented Mar 16, 2023

that's more work in terms of maintaining the mapping :), i'd prefer to control it from the calling side (the argument attribute) if there's a usecase

not validating the payload by default sounds like a footgun to me

combining multiple sources eg. body/query/cookie/header into a single DTO could be done witth a new attribute #[MapRequest] which takes a nested attribute mapping, using attributes such as #49134 or the one from this PR

then you can create pre-defined mappings in userland: MapMySpecialRequest extends MapRequest

it doesnt block this PR IMHO. please merge it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet