mirror of
https://github.com/doctrine/orm.git
synced 2026-03-23 22:42:18 +01:00
Incorrect commit order when inserting multiple entities #6331
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @AssortedParrot on GitHub (Oct 12, 2019).
Bug Report
Summary
In some cases, Doctrine will insert entities into the database in the wrong order, causing constraint violations for non-nullable foreign keys.
Current behavior
As a simple test case with two entities, consider an application where users can upload files: it would have a
Userand anUploadedFileentity.Other than its ID, an
UploadedFilehas two fields:owner(OneToOneUser, not nullable): so we can keep track of which user uploaded the file. This is not nullable because all files need to belong to a valid user.lastDownloadedBy(OneToOneUser, nullable): because for some reason we want to keep track of who last downloaded a file. This may be null in case nobody downloaded the file yet.A
Useronly has one field:lastUploadedFile(OneToOneUploadedFile, nullable): stores the last file the user uploaded, or null if the user didn't upload a file yetFrom the above mapping it can be seen that the
Usermust be inserted before theUploadedFilecan be inserted: noUploadedFilecan exist without aUser.In my test case, a
Userand anUploadedFileare created, and all fields are populated. Then,$em->persist()is called on theUploadedFileand then theUser. During the$em->flush(), Doctrine will try to insert theUploadedFilefirst, which will fail (because theownerfield is set toNULLsince theUseris not inserted yet,) causing an exception.How to reproduce
The test case mentioned above can be found in this repository. To reproduce it, clone the repository, run
composer installand then runphp test.php. It will create a SQLite database in the current directory, create the schema and then try to insert aUserand anUploadedFileobject into the database.The SQL logs show that Doctrine is trying to
INSERTtheUploadedFileobject first, which fails. It will then roll back the transaction and throw an exception.Expected behavior
I would expect the following behavior:
Userobject first (with thelastUploadedFilefield set toNULL)UploadedFileobject (theownerfield can now be set to the ID of theUserwe just inserted)Userobject (setlastUploadedFileto the ID of theUploadedFilewe just inserted)Additional information
It should be noted that I found many small things that, when changed, caused Doctrine to exhibit the expected behavior:
ownerandlastDownloadedByfields in theUploadedFileentity are defined$em->persist()lastDownloadedByfield to not nullableI believe this has something to do with the way the
CommitOrderCalculatortraverses the entity graph - changing the order of the fields or of the persist calls might cause it to walk the edges in a different order, resulting in a different commit order. Changing the nullability of the fields affects the weight that is assigned to the edges, which also has an effect on the commit order. However, I don't quite understand this part, so I was not able to investigate this further.The code in the test case repository uses SQLite, but I could also reproduce this on MySQL (MariaDB 10.3.18).
@SenseException commented on GitHub (Oct 15, 2019):
Thank you for reporting this issue and providing a reproducible code example. The nullable configuration in the mapping doesn't seem to influence the order and make the not null constraint happen. I'm not sure if this should be checked in
CommitOrderCalculatorthough. Do you have a suggestion on how/where to introduce a fix for this?@AssortedParrot commented on GitHub (Oct 17, 2019):
I'm not sure I understand. In my tests, changing the
lastDownloadedByfield inUploadedFileto not nullable will indeed change the commit order, avoiding the problem.Unfortunately, not really. I'm lacking a deeper understanding of the commit order calculation process. To be precise, I don't understand why
UnitOfWork#getCommitOrder()adds nullable associations as dependencies in theCommitOrderCalculator. Are nullable associations relevant for the commit order?If you try to
INSERTan entity that contains a nullable reference to another, not-yet-inserted entity, the reference can just be set toNULLduring the insertion, and then later updated once the referenced entity is inserted. (In fact, that's precisely what's happening with thelastUploadedFilefield if this issue is worked around.)So my first thought would be to only add non-nullable associations as dependencies in the
CommitOrderCalculator, but this approach causes a considerable number of tests to fail, so I did not pursue it further.And at this point I'm out of ideas. Would be nice if someone who knows more about the commit order calculation than me could chime in.
@Tetragramat commented on GitHub (May 5, 2020):
This bug exist at least 6 years and was reported multiple times #7006, #6499, #5538 and #4230
@kacperjurak commented on GitHub (Sep 5, 2020):
Today it was happened to me. Lost 3 hours. At the end just set column to nullable, but it's a workaround.
@kmsomebody commented on GitHub (Nov 14, 2020):
There are cases in which the only workaround is to call flush after each persist.
Bulk-updating one-to-one relations will fail due to the unique constraint, if not committed in the correct order.
This issue really needs a fix, very urgently. Calling flush multiple times is not an ideal solution.
@Cartman34 commented on GitHub (Jan 21, 2021):
I am having this issue in a simple case of ManyToOne relationship. Parent is required but child is inserted before with a null parent (but the parent is provided, I checked)
If A > B > C, A is referenced by B, C by B and child is owning relation (it contains the parent's id).
Parent does not contain any information about children, the relation is not reversed.
For my current case, the inserting order is C, A, B. It should be A, B, C.
I persisted data in the right order, each entity is persisted manually but Doctrine is using it wrong...
@rvanlaak commented on GitHub (May 20, 2021):
What about solely fixing this for the
OneToOnerelationships and cascade persist behavior? That could be a partial fix for the bigger problem, by having an impact that's as low as possible.Or, checking for such configurations on the join column and throw an exception with a more clear message that setting the joincolumn in specific cases is not supported? These cases will already lead to exceptions in the first place, so giving a clear error message would nog be a BC break either. That saves users from spending hours on debugging.
@mesolaries commented on GitHub (Jun 22, 2021):
Still having this issue in a simple
ManyToOnerelationship. I'm usingdoctrine/orm: 2.9.3. Any reasons why it's still not fixed?@rvanlaak commented on GitHub (Jun 22, 2021):
@mesolaries could you provide some minimal code for your entities on which the error occurs, so a unit test can get created to verify this problem (and thereafter a possible fix)? I've made an attempt for
OneToOnein #8703@mesolaries commented on GitHub (Jun 22, 2021):
Here's a minimal example code where error occurs:
Creating entities and flush:
Getting error on flush operation:
@rvanlaak
@rvanlaak commented on GitHub (Jun 30, 2021):
@mesolaries I've added the minimum of your code to PR #8703
Locally (when running on MySQL) these tests pass, so any idea why you got the exception?
@mesolaries commented on GitHub (Jun 30, 2021):
@rvanlaak I use Postgres 12. If you add an additional
flush()operation after persisting theApplicationentity (flushing 2 times in total), the errors disappear. In the example I use only oneflush()operation at the end. I think for some reason doctrine trying to insertApplicationPersonentity beforeApplicationentity.Doctrine INSERT order that causes the exception:
PersonentityApplicationPersonentity (Exception thrown ->nullvalue in columnapplication_id)Applicationentity (Not reaching)The correct INSERT order that should actually be:
PersonentityApplicationentityApplicationPersonentity withids ofPersonandApplicationentities@rvanlaak commented on GitHub (Jun 30, 2021):
The test has one flush: https://github.com/doctrine/orm/pull/8703/files#diff-31dcd3f7ddcb30c3ac344bcef0059e3e921a9f258de7c46d30b9faeff49a0e4aR49-R58
Can hereby verify that locally the same test also passes when running on Postgres.
@mesolaries commented on GitHub (Jun 30, 2021):
@rvanlaak I don't have any special cases that causes the error. @Cartman34 have the same issue with ManyToOne relationship: https://github.com/doctrine/orm/issues/7866#issuecomment-764557132
@rvanlaak commented on GitHub (Jul 21, 2021):
@Cartman34 does PR #8703 describe your scenario as well?
@mpdude commented on GitHub (Feb 27, 2023):
Please help: Give #10547 a try and let me know if it fixes this issue