Doctrine deletes all records when changing items in relation #5840

New Issue

admin · 2026-01-22T15:19:21+01:00

admin commented

2026-01-22 15:19:21 +01:00

Originally created by @Sharoff45 on GitHub (Jan 9, 2018).

Originally assigned to: @Ocramius on GitHub.

I have 2 entities:

Project
Issue

Each Project can have many Issues related to it.

I use this code to filter the Issues:


$filtered = []
foreach($project->getIssues() as $issue) {
   if(somethingIsTrue()) {
      $filtered[] = $issue;
      $issue->setAuthor('some-author');
   } else {
      $this->em->remove($issue);
   } 
}

$project->setIssues($filtered);
...
$this->em->flush();

In the end here is what I see in the log:

BEGIN TRANSACTION;
DELETE FROM issues WHERE project_id = ? [777];
UPDATE issues SET author = 'some-author' WHERE id = ? [123];
DELETE FROM issues WHERE id = ? [321];
COMMIT;

This output is not what I expected to see. I wouldn't expect it to delete all Issues within the Project. Could you please explain this behavior?

I also need to mention that this started happening only after we updated from Doctrine/ORM 2.5.14 to Doctrine/ORM 2.6.0

Originally created by @Sharoff45 on GitHub (Jan 9, 2018). Originally assigned to: @Ocramius on GitHub. I have 2 entities: 1. Project 2. Issue Each Project can have many Issues related to it. I use this code to filter the Issues: ~~~php $filtered = [] foreach($project->getIssues() as $issue) { if(somethingIsTrue()) { $filtered[] = $issue; $issue->setAuthor('some-author'); } else { $this->em->remove($issue); } } $project->setIssues($filtered); ... $this->em->flush(); ~~~ In the end here is what I see in the log: ~~~sql BEGIN TRANSACTION; DELETE FROM issues WHERE project_id = ? [777]; UPDATE issues SET author = 'some-author' WHERE id = ? [123]; DELETE FROM issues WHERE id = ? [321]; COMMIT; ~~~ This output is not what I expected to see. I wouldn't expect it to delete all Issues within the Project. Could you please explain this behavior? I also need to mention that this started happening only after we updated from Doctrine/ORM 2.5.14 to Doctrine/ORM 2.6.0

admin added the Bug Invalid labels 2026-01-22 15:19:21 +01:00

admin closed this issue

2026-01-22 15:19:21 +01:00

admin commented

2026-01-22 15:19:22 +01:00

@solexsylvain commented on GitHub (Jan 25, 2018):

I didn't expect such a regression when upgrading. My database in production was flushing order items when saving the order. I lost a whole day on fixing the mess it created. my fault, I should have had more behavioral tests in place. I almost cried. Now I'm laughing.

@solexsylvain commented on GitHub (Jan 25, 2018): I didn't expect such a regression when upgrading. My database in production was flushing order items when saving the order. I lost a whole day on fixing the mess it created. my fault, I should have had more behavioral tests in place. I almost cried. Now I'm laughing.

admin commented

2026-01-22 15:19:23 +01:00

@Ocramius commented on GitHub (Jan 25, 2018):

This looks like an orphan removal misconfiguration issue. Can you isolate this behaviour in a test case?

@Ocramius commented on GitHub (Jan 25, 2018): This looks like an orphan removal misconfiguration issue. Can you isolate this behaviour in a test case?

admin commented

2026-01-22 15:19:24 +01:00

@solexsylvain commented on GitHub (Jan 25, 2018):

I can probably set a test case but it will look the same as the code already posted up.
Do you need something in particular ? Yml file, small code block ?
When I go back to version 2.5.14 as mentioned by Sharoff45 who reported the bug, everything is awesome.

@solexsylvain commented on GitHub (Jan 25, 2018): I can probably set a test case but it will look the same as the code already posted up. Do you need something in particular ? Yml file, small code block ? When I go back to version 2.5.14 as mentioned by Sharoff45 who reported the bug, everything is awesome.

admin commented

2026-01-22 15:19:29 +01:00

@Ocramius commented on GitHub (Jan 25, 2018):

I can probably set a test case but it will look the same as the code already posted up.

It should be something runnable, like the tests in ddb3cd17bd/tests/Doctrine/Tests/ORM/Functional/Ticket

@Ocramius commented on GitHub (Jan 25, 2018): > I can probably set a test case but it will look the same as the code already posted up. It should be something runnable, like the tests in https://github.com/doctrine/doctrine2/tree/ddb3cd17bdc771e0d8ef1eb1b441f140e3b80cd9/tests/Doctrine/Tests/ORM/Functional/Ticket

admin commented

2026-01-22 15:19:29 +01:00

@Sharoff45 commented on GitHub (Jan 26, 2018):

I'm created pull request with test: https://github.com/doctrine/doctrine2/pull/7011

@Sharoff45 commented on GitHub (Jan 26, 2018): I'm created pull request with test: https://github.com/doctrine/doctrine2/pull/7011

admin commented

2026-01-22 15:19:31 +01:00

@Ocramius commented on GitHub (Jan 26, 2018):

Thanks!

@Ocramius commented on GitHub (Jan 26, 2018): Thanks!

admin commented

2026-01-22 15:19:32 +01:00

@Ocramius commented on GitHub (Jan 26, 2018):

@Sharoff45 @solexsylvain see #7011 discussion - this is indeed behaving as per design :-\

@Ocramius commented on GitHub (Jan 26, 2018): @Sharoff45 @solexsylvain see #7011 discussion - this is indeed behaving as per design :-\

admin commented

2026-01-22 15:19:34 +01:00

@solexsylvain commented on GitHub (Jan 26, 2018):

Hi Ocramius and Sharoff45,

I read it. I understand it was the intention at first but it was working "correctly" until 2.5.14.
I'm guessing you'll see more issue with this as Symfony seems to do the same as the setIssues at least in the case I'm in.
I'm using a form with child forms (order and its items). I do not allow adding or removing to the collection, just updating the items if need be.

I've tracked it down through $form->handleRequest($request) --> submit() -> ... -> PropertyAccessor.php(writeProperty) where you can see "$object->{$access[self::ACCESS_NAME]}($value);". It took sometime to get there, the level of abstraction in Symfony is big.

This is just not something people will expect when upgrading. It's a breaking change even if it would correct an existing flaw (comparing to intentional design). It's a minor version change.

In production, in a small scale I had to correct about 50 orders that lost their items (I was able to stop the business from doing operations on the system within 1 hour). I was lucky enough that it was happening when closing the order. I was able to refer to a movement table (inventory related) to rebuild the data. If I didn't have that movement table, having a backup run every 3 hours on that db and had replication enabled didn't do any good to me. I can't imagine what kind of damage it would do at a large scale.

@solexsylvain commented on GitHub (Jan 26, 2018): Hi Ocramius and Sharoff45, I read it. I understand it was the intention at first but it was working "correctly" until 2.5.14. I'm guessing you'll see more issue with this as Symfony seems to do the same as the setIssues at least in the case I'm in. I'm using a form with child forms (order and its items). I do not allow adding or removing to the collection, just updating the items if need be. I've tracked it down through $form->handleRequest($request) --> submit() -> ... -> PropertyAccessor.php(writeProperty) where you can see "$object->{$access[self::ACCESS_NAME]}($value);". It took sometime to get there, the level of abstraction in Symfony is big. This is just not something people will expect when upgrading. It's a breaking change even if it would correct an existing flaw (comparing to intentional design). It's a minor version change. In production, in a small scale I had to correct about 50 orders that lost their items (I was able to stop the business from doing operations on the system within 1 hour). I was lucky enough that it was happening when closing the order. I was able to refer to a movement table (inventory related) to rebuild the data. If I didn't have that movement table, having a backup run every 3 hours on that db and had replication enabled didn't do any good to me. I can't imagine what kind of damage it would do at a large scale.

admin commented

2026-01-22 15:19:35 +01:00

@solexsylvain commented on GitHub (Jan 26, 2018):

It's not to complain, I just want to avoid it for other people.

@solexsylvain commented on GitHub (Jan 26, 2018): It's not to complain, I just want to avoid it for other people.

admin commented

2026-01-22 15:19:35 +01:00

@Ocramius commented on GitHub (Jan 26, 2018):

I've tracked it down through $form->handleRequest($request) --> submit() -> ... -> PropertyAccessor.php(writeProperty) where you can see "$object->{$access[self::ACCESS_NAME]}($value);". It took sometime to get there, the level of abstraction in Symfony is big.

This is not an abstraction, this is the typical symfony (and other frameworks, some of which I've built myself too, sorry!) madness.

I understand what is going on, but the behavior was supposed to be removal of any orphaned entities, and replacing the collection does indeed orphan them.

This has been expected since orphanRemoval was introduced, and caused quite a fuss in similar form components too, for example:

57c0ee9586/src/DoctrineModule/Stdlib/Hydrator/DoctrineObject.php (L379-L394)

So I think this needs to be fixed somehow inside PropertyAccess or the symfony form component.

@Ocramius commented on GitHub (Jan 26, 2018): > I've tracked it down through $form->handleRequest($request) --> submit() -> ... -> PropertyAccessor.php(writeProperty) where you can see "$object->{$access[self::ACCESS_NAME]}($value);". It took sometime to get there, the level of abstraction in Symfony is big. This is not an abstraction, this is the typical symfony (and other frameworks, some of which I've built myself too, sorry!) madness. I understand what is going on, but the behavior was supposed to be removal of any orphaned entities, and replacing the collection does indeed orphan them. This has been expected since `orphanRemoval` was introduced, and caused quite a fuss in similar form components too, for example: https://github.com/doctrine/DoctrineModule/blob/57c0ee958664e3a04b211607fd21b7e6b6fc4e68/src/DoctrineModule/Stdlib/Hydrator/DoctrineObject.php#L379-L394 So I think this needs to be fixed somehow inside PropertyAccess or the symfony form component.

admin commented

2026-01-22 15:19:37 +01:00

@solexsylvain commented on GitHub (Jan 26, 2018):

Symfony is great and I understand you don't get to that point without adding complexity to the code.
Doctrine is the best way that I was able to access the data and map it to objects all languages considered. If you do some .Net at some point, look at dapper, it's interesting and fast.

I know I currently don't have the knowledge of the tools to come up with a fix for that. I just know it's in the wild. If there is a way I could help fix it, let me know and I'll try.

@solexsylvain commented on GitHub (Jan 26, 2018): Symfony is great and I understand you don't get to that point without adding complexity to the code. Doctrine is the best way that I was able to access the data and map it to objects all languages considered. If you do some .Net at some point, look at dapper, it's interesting and fast. I know I currently don't have the knowledge of the tools to come up with a fix for that. I just know it's in the wild. If there is a way I could help fix it, let me know and I'll try.

admin commented

2026-01-22 15:19:39 +01:00

@Ocramius commented on GitHub (Jan 26, 2018):

@solexsylvain if you take the snippet I reported above (from DoctrineModule) and report an issue against symfony/symfony, I'm sure a fix (and useful discussion) can arise

@Ocramius commented on GitHub (Jan 26, 2018): @solexsylvain if you take the snippet I reported above (from `DoctrineModule`) and report an issue against `symfony/symfony`, I'm sure a fix (and useful discussion) can arise

admin commented

2026-01-22 15:19:39 +01:00

@solexsylvain commented on GitHub (Jan 26, 2018):

It's done. The issue is at https://github.com/symfony/symfony/issues/25937
Feel free to join the conversation. Not sure If what I wrote is easy to understand.
Thanks a lot for your help.

@solexsylvain commented on GitHub (Jan 26, 2018): It's done. The issue is at https://github.com/symfony/symfony/issues/25937 Feel free to join the conversation. Not sure If what I wrote is easy to understand. Thanks a lot for your help.

admin commented

2026-01-22 15:19:40 +01:00

@lcobucci commented on GitHub (Feb 18, 2018):

@solexsylvain @Ocramius can we close this issue then?

@lcobucci commented on GitHub (Feb 18, 2018): @solexsylvain @Ocramius can we close this issue then?

admin commented

2026-01-22 15:19:40 +01:00

@Ocramius commented on GitHub (Feb 19, 2018):

Closing as invalid here - further discussion continued at symfony/symfony#25937

This behavior could possibly be improved in 3.x by diffing replaced collections instead of marking everything as an orphan removal, but this cannot be changed in 2.x due to BC constraints.

@Ocramius commented on GitHub (Feb 19, 2018): Closing as `invalid` here - further discussion continued at symfony/symfony#25937 This behavior could possibly be improved in `3.x` by diffing replaced collections instead of marking everything as an orphan removal, but this cannot be changed in `2.x` due to BC constraints.

admin commented

2026-01-22 15:19:41 +01:00

@solexsylvain commented on GitHub (Feb 19, 2018):

I'll continue on the symfony board. I still have to reply, not convinced making the property public is such a good idea and providing my own api to do as stated by stof is counter productive when there could be a lasting fix in doctrine in the future. Thanks for caring about this issue. I'll be patient for version 3.

@solexsylvain commented on GitHub (Feb 19, 2018): I'll continue on the symfony board. I still have to reply, not convinced making the property public is such a good idea and providing my own api to do as stated by stof is counter productive when there could be a lasting fix in doctrine in the future. Thanks for caring about this issue. I'll be patient for version 3.

admin commented

2026-01-22 15:19:41 +01:00

@alucic commented on GitHub (May 31, 2018):

So 2.6 should have been tagged 3.0 since it has huge breaking change. Suddenly, code that was working now not only doesn't work anymore, but deletes your data.

https://github.com/doctrine/doctrine2/pull/7011#issuecomment-360717979

Tests correctly reflect the issue report in #6976, but the ORM is behaving as per design here, since a collection with orphanRemoval on it is being replaced.

This is a bug, it's deleting data without any warnings and should be fixed. It's a breaking change and this "design decision" is for 3.0 not 2.6

@alucic commented on GitHub (May 31, 2018): So 2.6 should have been tagged 3.0 since it has huge breaking change. Suddenly, code that was working now not only doesn't work anymore, but deletes your data. https://github.com/doctrine/doctrine2/pull/7011#issuecomment-360717979 > Tests correctly reflect the issue report in #6976, but the ORM is behaving as per design here, since a collection with orphanRemoval on it is being replaced. This is a bug, it's deleting data without any warnings and should be fixed. It's a breaking change and this "design decision" is for 3.0 not 2.6

admin commented

2026-01-22 15:19:41 +01:00

@mattsah commented on GitHub (Apr 11, 2019):

This looks very similar to an issue I'm having. It's not clear why replacing a collection with a new collection of the same (or modified) records counts as those records being orphaned. The records are not orphaned, they're just in a different container.

@mattsah commented on GitHub (Apr 11, 2019): This looks very similar to an issue I'm having. It's not clear why replacing a collection with a new collection of the same (or modified) records counts as those records being orphaned. The records are not orphaned, they're just in a different container.

admin commented

2026-01-22 15:19:41 +01:00

@acasademont commented on GitHub (Jul 1, 2019):

I was hit by this today, quite painful to debug. I was using the "filter" method on the collection, which was recreating the collection with the new elements instead of removing the elements from the existing collection and thus marking all its elements as "orphaned" even though they were not. I hope Doctrine 3 can be a bit smarter and actually diff the collections

@acasademont commented on GitHub (Jul 1, 2019): I was hit by this today, quite painful to debug. I was using the "filter" method on the collection, which was recreating the collection with the new elements instead of removing the elements from the existing collection and thus marking all its elements as "orphaned" even though they were not. I hope Doctrine 3 can be a bit smarter and actually diff the collections

admin commented

2026-01-22 15:19:41 +01:00

@ZaneCEO commented on GitHub (Jun 12, 2020):

I've been hit by this. For those looking fo a quick solution: remove orphanRemoval=true from the equivalent of the Project entity and it start to work as expected.

Edit for clarity: I understand that this is not a great solution design-wise, but it's effective for the specific problem at hand.

@ZaneCEO commented on GitHub (Jun 12, 2020): I've been hit by this. For those looking fo a quick solution: remove `orphanRemoval=true` from the equivalent of the `Project` entity and it start to work as expected. Edit for clarity: I understand that this is not a great solution design-wise, but it's effective for the specific problem at hand.

admin commented

2026-01-22 15:19:42 +01:00

@s001dxp commented on GitHub (Oct 19, 2021):

This caused me to lose so much data. We had to stop working with the application that we spent years developing and use excel instead. It took months upons months to finally track it orphanRemoval.

Things like this causes a person to lose trust in the packages they use when a breaking change is introduced in a minor release.

@s001dxp commented on GitHub (Oct 19, 2021): This caused me to lose so much data. We had to stop working with the application that we spent years developing and use excel instead. It took months upons months to finally track it orphanRemoval. Things like this causes a person to lose trust in the packages they use when a breaking change is introduced in a minor release.

admin referenced this issue

2026-01-22 16:05:19 +01:00

[PR #5840] Move performance tests to phpbench/phpbench based benchmarks #9754

admin referenced this issue

2026-01-22 16:06:14 +01:00

[PR #6575] Move performance tests to phpbench #10041

Sign in to join this conversation.