Deprecate Partial Objects #6629

@AlexisFinn commented on GitHub (Mar 11, 2022):

Hello, just to be clear, do you plan on removing partial arrays, as in

$qb->selec('partial  user.{id, name}')
->getQuery()->getArrayResult();

Because that's actually really usefull for retrieving a nicely formatted array.

@AlexisFinn commented on GitHub (Mar 11, 2022): Hello, just to be clear, do you plan on removing partial arrays, as in ```php $qb->selec('partial user.{id, name}') ->getQuery()->getArrayResult(); ``` Because that's actually really usefull for retrieving a nicely formatted array.

admin commented

@raziel057 commented on GitHub (Aug 21, 2023):

I don't see any answer for the question raised by AlexisFinn

Can you reconsider the use of partial with Array Hydration ? I understand the issues given for Partial Object Hydration but there is no problem at all working with array. it seems better and less time consuming than having to define a Dto or having to use a scalar hydration with the need to specify each property and having all in a flat array.

Ex.:

$qb->select('d, p, partial f.{id, mimetype, name, size}')
            ->from(Document::class, 'd')
            ->join('d.file', 'f')
            ->leftJoin('d.publications', 'p')
            ->getQuery()->getArrayResult();

@raziel057 commented on GitHub (Aug 21, 2023): I don't see any answer for the question raised by [AlexisFinn](https://github.com/AlexisFinn) Can you reconsider the use of partial with Array Hydration ? I understand the issues given for Partial Object Hydration but there is no problem at all working with array. it seems better and less time consuming than having to define a Dto or having to use a scalar hydration with the need to specify each property and having all in a flat array. Ex.: ```php $qb->select('d, p, partial f.{id, mimetype, name, size}') ->from(Document::class, 'd') ->join('d.file', 'f') ->leftJoin('d.publications', 'p') ->getQuery()->getArrayResult(); ``` ![image](https://github.com/doctrine/orm/assets/652505/2436bfa3-025a-4009-8ee9-eda3ff3b8e81)

admin commented

@raziel057 commented on GitHub (Aug 21, 2023):

From Doctrine page https://www.doctrine-project.org/projects/doctrine-orm/en/2.16/reference/partial-objects.html it's mentioned:

The partial object problem in general does not apply to methods or queries where you do not retrieve the query result as objects. Examples are: Query#getArrayResult(), Query#getScalarResult(), Query#getSingleScalarResult(), etc.

@raziel057 commented on GitHub (Aug 21, 2023): From Doctrine page https://www.doctrine-project.org/projects/doctrine-orm/en/2.16/reference/partial-objects.html it's mentioned: > The partial object problem in general does not apply to methods or queries where you do not retrieve the query result as objects. Examples are: Query#getArrayResult(), Query#getScalarResult(), Query#getSingleScalarResult(), etc.

admin commented

@beberlei commented on GitHub (Nov 1, 2023):

@gharlan @pszalko for multi step hydration the new way to go will be https://github.com/doctrine/orm/pull/8391 this is essentially what @stepozer mentioned laravel also does.

As for array hydration with partial, this is something we haven't considered before, so its a valid use-case and I had the idea maybe this works with a function that can only be used in combination with array hydration:

SELECT PARTIAL(p, id, name, description) FROM product p

@beberlei commented on GitHub (Nov 1, 2023): @gharlan @pszalko for multi step hydration the new way to go will be https://github.com/doctrine/orm/pull/8391 this is essentially what @stepozer mentioned laravel also does. As for array hydration with partial, this is something we haven't considered before, so its a valid use-case and I had the idea maybe this works with a function that can only be used in combination with array hydration: ```sql SELECT PARTIAL(p, id, name, description) FROM product p ```

admin commented

@bakugo commented on GitHub (Nov 30, 2023):

@beberlei #8391 is not an adequate replacement because it does not give the user the same control over when and how the hydration happens. I need to be able to hydrate the associations I need, when I need them, in the queries I specify, so if PARTIAL is removed I'll be forced to just load already loaded entities multiple times for no reason.

Removing partial objects is understandable but there still should be some way to signal to DQL "only load the ID for this alias for association purposes because the relevant entities were already previously loaded", it could even throw an exception if it encounters an ID that isn't already loaded.

@bakugo commented on GitHub (Nov 30, 2023): @beberlei #8391 is not an adequate replacement because it does not give the user the same control over when and how the hydration happens. I need to be able to hydrate the associations I need, when I need them, in the queries I specify, so if PARTIAL is removed I'll be forced to just load already loaded entities multiple times for no reason. Removing partial objects is understandable but there still should be some way to signal to DQL "only load the ID for this alias for association purposes because the relevant entities were already previously loaded", it could even throw an exception if it encounters an ID that isn't already loaded.

admin commented

2026-01-22 15:36:08 +01:00

@klkvsk commented on GitHub (Dec 26, 2023):

After reading all comments and links in this issue, it's still unclear how to load entities partially.
Referencing the example by @raziel057

$qb->select('d, p, partial f.{id, mimetype, name, size}')
            ->from(Document::class, 'd')
            ->join('d.file', 'f')
            ->leftJoin('d.publications', 'p')
            ->getQuery()->  getResult() // objects, not array!

So let's assume we want to get documents as objects with all relations, but there is a file.content field, that can be lots of data we won't ever need in that result, and we want to omit just it.

What @beberlei suggested

"SELECT new MyDTO()" -- it would not be possible to hydrate $document->file property, which is of class File, with a some other DTO object, isn't it?
"fetch=Eager" -- but fetch mode is applied on a related entity(-ies) as a whole, not their separate properties, isn't it?
"Add support for lazy loading some properties" -- well, sounds like that's it, but still not implemented.

So what do you actually suggest, except ignoring the deprecation warning until (3) is merged?

@klkvsk commented on GitHub (Dec 26, 2023): After reading all comments and links in this issue, it's still unclear how to load entities partially. Referencing the example by @raziel057 > ``` > $qb->select('d, p, partial f.{id, mimetype, name, size}') > ->from(Document::class, 'd') > ->join('d.file', 'f') > ->leftJoin('d.publications', 'p') > ->getQuery()-> getResult() // objects, not array! > ``` So let's assume we want to get documents as objects with all relations, but there is a `file.content` field, that can be lots of data we won't ever need in that result, and we want to omit just it. What @beberlei suggested 1. "SELECT new MyDTO()" -- it would not be possible to hydrate $document->file property, which is of class File, with a some other DTO object, isn't it? 2. "fetch=Eager" -- but fetch mode is applied on a related entity(-ies) as a whole, not their separate properties, isn't it? 3. "Add support for lazy loading some properties" -- well, sounds like that's it, but still not implemented. So what do you actually suggest, except ignoring the deprecation warning until (3) is merged?

admin commented

@beberlei commented on GitHub (Dec 26, 2023):

@klkvsk yes, ignore the deprecation and stay on 2.x - you can call Deprecation::ignoreDeprecation (or similar) with the url identifer to silence it for now

@beberlei commented on GitHub (Dec 26, 2023): @klkvsk yes, ignore the deprecation and stay on 2.x - you can call Deprecation::ignoreDeprecation (or similar) with the url identifer to silence it for now

admin commented

2026-01-22 15:36:08 +01:00

@beberlei commented on GitHub (Dec 26, 2023):

@bakugo you can set the assoc lazy by default and use Query::setFetchMode to change on a per query basis. This is the same level of granularity that PARTIAL offers

@beberlei commented on GitHub (Dec 26, 2023): @bakugo you can set the assoc lazy by default and use Query::setFetchMode to change on a per query basis. This is the same level of granularity that PARTIAL offers

admin commented

@bakugo commented on GitHub (Dec 26, 2023):

@beberlei FETCH_EAGER is not adequate because it runs an extra query for each association.

The use case in question is basically this: https://github.com/doctrine/orm/issues/4762#issuecomment-1350722383. I have one "main" entity that has a lot of fields and a lot of associations. If I use EAGER, I end up with more queries than necessary (and it cannot handle nested associations apparently). If I simply JOIN all the associations in one query, there will be too many of them and the query will be extremely slow and inefficient.

The only approach that reliably works is splitting the JOINs into multiple queries, aka multi-step hydration, where you select the "main" entity by ID multiple times and JOIN a different set of associations each time, such that each individual query is efficient, and you end up with all the associations you need hydrated in a low number of queries. The only real problem here is that each query will waste time selecting the fields of the main entity that were already loaded before. SELECT PARTIAL entity.{id} solves this, and does NOT result in partially loaded objects because those objects were already loaded, so there's no reason for it to not be allowed in this case.

@bakugo commented on GitHub (Dec 26, 2023): @beberlei FETCH_EAGER is not adequate because it runs an extra query for each association. The use case in question is basically this: https://github.com/doctrine/orm/issues/4762#issuecomment-1350722383. I have one "main" entity that has a lot of fields and a lot of associations. If I use EAGER, I end up with more queries than necessary (and it cannot handle nested associations apparently). If I simply JOIN all the associations in one query, there will be too many of them and the query will be extremely slow and inefficient. The only approach that reliably works is splitting the JOINs into multiple queries, aka [multi-step hydration](https://ocramius.github.io/blog/doctrine-orm-optimization-hydration/), where you select the "main" entity by ID multiple times and JOIN a different set of associations each time, such that each individual query is efficient, and you end up with all the associations you need hydrated in a low number of queries. The only real problem here is that each query will waste time selecting the fields of the main entity that were already loaded before. `SELECT PARTIAL entity.{id}` solves this, and does NOT result in partially loaded objects because those objects were already loaded, so there's no reason for it to not be allowed in this case.

admin commented

@beberlei commented on GitHub (Dec 26, 2023):

The inefficent way hydration for multiple to many assocs work, i could imagine one query per assoc will be faster most of the time.

@beberlei commented on GitHub (Dec 26, 2023): The inefficent way hydration for multiple to many assocs work, i could imagine one query per assoc will be faster most of the time.

admin commented

@clesquir commented on GitHub (Feb 21, 2024):

As for array hydration with partial, this is something we haven't considered before, so its a valid use-case and I had the idea maybe this works with a function that can only be used in combination with array hydration:
SELECT PARTIAL(p, id, name, description) FROM product p

We are using partials a lot in our application in order to return nicely formatted arrays. Converting all of them with DTOs will take weeks of work. Also, we need to include OneToMany in there which I was not able to achieve using a DTO.

Would this function be available in Doctrine 3.0 or is it only an idea?

If this is only an idea, can you provide some pointers on how to implement?

Thanks!

@clesquir commented on GitHub (Feb 21, 2024): > As for array hydration with partial, this is something we haven't considered before, so its a valid use-case and I had the idea maybe this works with a function that can only be used in combination with array hydration: > > ```sql > SELECT PARTIAL(p, id, name, description) FROM product p > ``` We are using partials a lot in our application in order to return nicely formatted arrays. Converting all of them with DTOs will take weeks of work. Also, we need to include OneToMany in there which I was not able to achieve using a DTO. Would this function be available in Doctrine 3.0 or is it only an idea? If this is only an idea, can you provide some pointers on how to implement? Thanks!

admin commented

@lowwa132 commented on GitHub (Mar 5, 2024):

Same issue as @clesquir here, the DTO alternative is not suitable for OneToMany relations usecase.

@lowwa132 commented on GitHub (Mar 5, 2024): Same issue as @clesquir here, the DTO alternative is not suitable for OneToMany relations usecase.

admin commented

2026-01-22 15:36:10 +01:00

@beberlei commented on GitHub (Mar 16, 2024):

After long deliberation I see no other way than allowing PARTIAL keyword again in 3.0, but only in ArrayHydrator. We will work on adding this again in ORM 3.2 (as 3.0 and 3.1 are already released).

We will also change the deprecation in 2.x to only trigger when the PARTIAL keyword is used in combination with a hydrator that calls UnitOfWork::createEntity.

This will keep the keyword as long as no alternative API for this use-case is available, such as the Entity Graph API from JPA, https://www.baeldung.com/jpa-entity-graph#2-defining-an-entity-graph-with-the-jpa-api

This issue title and description will be updated accordingly once #11365 and #11366 are merged.

@beberlei commented on GitHub (Mar 16, 2024): After long deliberation I see no other way than allowing PARTIAL keyword again in 3.0, **but only in ArrayHydrator**. We will work on adding this again in ORM 3.2 (as 3.0 and 3.1 are already released). We will also change the deprecation in 2.x to only trigger when the PARTIAL keyword is used in combination with a hydrator that calls `UnitOfWork::createEntity`. This will keep the keyword as long as no alternative API for this use-case is available, such as the Entity Graph API from JPA, https://www.baeldung.com/jpa-entity-graph#2-defining-an-entity-graph-with-the-jpa-api This issue title and description will be updated accordingly once #11365 and #11366 are merged.

admin commented

@fmonts commented on GitHub (Jul 8, 2024):

I use partial in Symfony forms, like this:

->add('allowedUsers', EntityType::class, [
    'class' => User::class,
    'query_builder' => function (EntityRepository $er) use ($options) {
        return $er->createQueryBuilder('u')
            ->select('partial u.{id, username}, partial up.{user}, partial us.{user}')
            ->leftJoin('u.profile', 'up')
            ->leftJoin('u.settings', 'us')
            [...]
            ->orderBy('u.username', 'ASC');
    },
    'label' => 'Allowed users',
    'required' => false,
    'multiple' => true,
    'choice_label' => function (User $choice) {
        return $choice->getId() . ' - ' . $choice->getUsername();
    }
])

(profile and settings are a one-to-one, I need to join them otherwise it will do n queries to fetch them instead of 1, where n is the number of users)

Since I only need ID + username to display them in a dropdown, it's a nice optimization to improve speed and reduce the traffic from the db.

See the difference in performance, 42 ms vs 307:

Why deprecating this? I don't get it

@fmonts commented on GitHub (Jul 8, 2024): I use `partial` in Symfony forms, like this: ``` ->add('allowedUsers', EntityType::class, [ 'class' => User::class, 'query_builder' => function (EntityRepository $er) use ($options) { return $er->createQueryBuilder('u') ->select('partial u.{id, username}, partial up.{user}, partial us.{user}') ->leftJoin('u.profile', 'up') ->leftJoin('u.settings', 'us') [...] ->orderBy('u.username', 'ASC'); }, 'label' => 'Allowed users', 'required' => false, 'multiple' => true, 'choice_label' => function (User $choice) { return $choice->getId() . ' - ' . $choice->getUsername(); } ]) ``` (profile and settings are a one-to-one, I need to join them otherwise it will do n queries to fetch them instead of 1, where n is the number of users) Since I only need ID + username to display them in a dropdown, it's a nice optimization to improve speed and reduce the traffic from the db. See the difference in performance, 42 ms vs 307: ![image](https://github.com/doctrine/orm/assets/8755314/fb988283-ec52-4154-8926-e2d6cdba3edf) Why deprecating this? I don't get it

admin commented

2026-01-22 15:36:10 +01:00

@derrabus commented on GitHub (Jul 8, 2024):

Since I only need ID + username to display them in a dropdown, it's a nice optimization to improve speed and reduce the traffic from the db.

Why do you hydrate objects anyway if you only need two scalar values?

@derrabus commented on GitHub (Jul 8, 2024): > Since I only need ID + username to display them in a dropdown, it's a nice optimization to improve speed and reduce the traffic from the db. Why do you hydrate objects anyway if you only need two scalar values?

admin commented

2026-01-22 15:36:10 +01:00

@idybil commented on GitHub (Jul 9, 2024):

Since I only need ID + username to display them in a dropdown, it's a nice optimization to improve speed and reduce the traffic from the db.

Why do you hydrate objects anyway if you only need two scalar values?

Is this because of the EntityType which is expecting an entity ? so the persisting of the entity is managed automatically by the form.

@idybil commented on GitHub (Jul 9, 2024): > > Since I only need ID + username to display them in a dropdown, it's a nice optimization to improve speed and reduce the traffic from the db. > > Why do you hydrate objects anyway if you only need two scalar values? Is this because of the EntityType which is expecting an entity ? so the persisting of the entity is managed automatically by the form.

admin commented

@derrabus commented on GitHub (Jul 9, 2024):

I see. And if you select an entity with this form, you would also get a partially hydrated one as a result, right?

Can you please open an issue on the Symfony tracker and link to this issue here? Maybe we can improve the EntityType so you don't need to use partial hydration anymore. But discussing that in this issue would be at bit out of scope.

@derrabus commented on GitHub (Jul 9, 2024): I see. And if you select an entity with this form, you would also get a partially hydrated one as a result, right? Can you please open an issue on the Symfony tracker and link to this issue here? Maybe we can improve the `EntityType` so you don't need to use partial hydration anymore. But discussing that in this issue would be at bit out of scope.

admin commented

@idybil commented on GitHub (Jul 9, 2024):

I see. And if you select an entity with this form, you would also get a partially hydrated one as a result, right?

Yes

Can you please open an issue on the Symfony tracker and link to this issue here? Maybe we can improve the EntityType so you don't need to use partial hydration anymore. But discussing that in this issue would be at bit out of scope.

@fmonts , do you have time to open an issue on the Symfony tracker ?

@idybil commented on GitHub (Jul 9, 2024): > I see. And if you select an entity with this form, you would also get a partially hydrated one as a result, right? Yes > Can you please open an issue on the Symfony tracker and link to this issue here? Maybe we can improve the `EntityType` so you don't need to use partial hydration anymore. But discussing that in this issue would be at bit out of scope. @fmonts , do you have time to open an issue on the Symfony tracker ?

admin commented

@cbichis commented on GitHub (Jul 21, 2024):

@beberlei maybe we can still keep partial also when using \Doctrine\ORM\Query::HINT_READ_ONLY ? Not just for array hydration?

@cbichis commented on GitHub (Jul 21, 2024): @beberlei maybe we can still keep partial also when using \Doctrine\ORM\Query::HINT_READ_ONLY ? Not just for array hydration?

admin commented

@kevinpapst commented on GitHub (Aug 19, 2024):

I was trying to tackle a few deprecated PARTIALS today, but found that the status quo is not yet in feature parity with the PARTIAL construct. Reading through several issues, I see that many others run into the same problems.

My main 3 issues which prevent me from upgrading to ORM v3 are:

better support for ManyToMany collections (currently O(n) as 1 query per entity is triggered)
support for multi-level hydration
support for hydrating collections of already initialized entities (least important, would minimize refactoring time)

Is there any consideration to tackle the missing points?

@kevinpapst commented on GitHub (Aug 19, 2024): I was trying to tackle a few deprecated PARTIALS today, but found that the status quo is not yet in feature parity with the PARTIAL construct. Reading through several issues, I see that many others run into the same problems. My main 3 issues which prevent me from upgrading to ORM v3 are: - better support for ManyToMany collections (currently O(n) as 1 query per entity is triggered) - support for [multi-level hydration](https://github.com/doctrine/orm/issues/11254) - support for hydrating collections of [already initialized entities](https://github.com/doctrine/orm/issues/10314) (least important, would minimize refactoring time) Is there any consideration to tackle the missing points?

admin commented