* Do not eagerly set metadata from ResolveTargetEntityListener
When using the `ResolveTargetEntityListener` to substitute `targetEntities` in association mappings, do not eagerly put the resolved (target) entity into the class metadata cache under the class name of the original entity.
#### Motivation
I have a library that wants to distribute a MappedSuperclass as the base for some functionality. It will be necessary that clients using the library will extend the MappedSuperclass to fill in some blanks, creating the first real `#[Entity]` instance of it.
This client-provided entity will be the primary means of working with the class. Thus, I was following the [note in the documentation](https://www.doctrine-project.org/projects/doctrine-orm/en/3.5/reference/inheritance-mapping.html#mapped-superclasses:~:text=It%20is%2C%20however) and using the `ResolveTargetEntityListener` to declare that whenever an association refers to that mapped superclass, the particular entity class shall be used instead.
> One-To-Many associations are not generally possible on a mapped superclass, since they require the "many" side to hold the foreign key.
> It is, however, possible to use the [ResolveTargetEntityListener](https://www.doctrine-project.org/projects/doctrine-orm/en/3.5/cookbook/resolve-target-entity-listener.html) to replace references to a mapped superclass with an entity class at runtime. As long as there is only one entity subclass inheriting from the mapped superclass and all references to the mapped superclass are resolved to that entity class at runtime, the mapped superclass can use One-To-Many associations and be named as the targetEntity on the owning sides.
#### Changes made
The `ResolveTargetEntityListener` primarily does what its name suggests: For newly loaded class metadata, it inspects all associations declared and replaces the `targetEntity` with new (resolved) values.
But additionally, when a loaded class is the target of such a resolution, it would also put the class metadata into the cache under the name of the original entity.
I think that extra step is wrong, and this PR removes it. It had the side effect that when other classes extending the MappedSuperclass were loaded _after_ the resolve target class has been seen for the first time, the metadata for those classes would not inherit from the mapped superclass anymore, but from the target entity class instead. In my real-life use case, this causes weird mapping errors down the road; as of ^3.0, it would throw a mapping exception asking to configure inheritance mapping. But note that there would be no inheritance between the two entity classes at all.
#### More background
The documentation [describes the use of `ResolveTargetEntityListener`](https://www.doctrine-project.org/projects/doctrine-orm/en/3.5/cookbook/resolve-target-entity-listener.html) with an interface that is resolved to an entity class. For an interface, adding the extra metadata does not make a difference, since it never interferes with actual entity or mapped superclasses.
The initial idea of adding a copy of the entity class metadata under the interface name came from commit
9c7f3f2747 in #385. The goal was to make it possible to also find entities by interface names, like so:
```
$em->find('Foo\BarBundle\Entity\PersonInterface', 1);
```
It then [turned out that this only worked when the resolution had already been applied](https://github.com/doctrine/orm/pull/385#issuecomment-6658893). So, the new `onClassMetadataNotFound` event was added and the resolution map would be checked in that case as well (#1181). The inital code stayed in place, possibly giving a small performance gain.
In my real-world use case and the test case I added in this PR, the associations are even self-referencing. That should not really be necessary for the problem to surface. I decided to keep it this way to show that the `targetEntity` need not be an interface after all, and that a MappedSuperclass can be used in the same way.
This PR fixes the `Criteria` matching API for `IN` and `NIN` conditions with values that are arrays, by making sure that type information for the matched field is passed to the DBAL level correctly.
Passing the right parameter type to DBAL is important to make sure parameter conversions are applied before matching at the database level.
Memory-based collections (`ArrayCollection`s or initialized collection fields) would perform matching on the objects in memory where no type conversion to the database representation is required, giving correct results.
But uninitialized collections that have their conditions evaluated at the database level need to convert parameter values to the database representation before performing the comparison.
One extra challenge is that the DBAL type system does currently not support array-valued parameters for custom types. Only a [limited list of types](https://www.doctrine-project.org/projects/doctrine-dbal/en/4.2/reference/data-retrieval-and-manipulation.html#list-of-parameters-conversion) is supported.
I discussed this with @morozov at the Doctrine Hackathon and came to the conclusion that it would be best to work around this limitation at the ORM level. Thus, this fix recognizes array-valued parameters and creates multiple placeholders (like `?, ?, ?`) for them, flattening out the arrays in the parameter list and repeating the type information for each one of them.
Previous stalled attempt to fix this was in #11897.
Fix JoinedSubclassPersister as BasicEntityPersister was already fixed in GH-10735.
The fix can be verified by modifying UnitOfWork to execute `BasicEntityPersister::executeInserts()` for multiple entities at once for the same entity class/persister instance - https://github.com/doctrine/orm/blob/2.20.3/src/UnitOfWork.php#L1186 - then reproducible on `Doctrine\Tests\ORM\Functional\Ticket\GH10531Test::testInserts` test.
As extending/modifying UnitOfWork in tests in not easily possible, I submit this fix for v2.x without a test.
When the persister is extended to do a multi update, the caching is not
wanted. The impact is minimal as the CPU/time overhead per query is
much bigger and the prepared statement is not cached anyway.
This command's purpose is to provide structured data, except for a call
to caution() that warns the user in case they do not have any mapped
entities or they have errors.
Symfony 7.3 is not available to all of our users, so we cannot switch to
native lazy objects, which require a PHP version higher than the lowest
PHP version we support.
In f256d996cc, I did a global move to
stderr for notifications, and went a bit overboard for
MappingDescribeCommand, which purpose is to output a description.
For some reason, it does not appear to work when nested inside a
code-block directive. Anyway, if you specify the language attribute, you
get markup identical to what you obtain when using code-block and
literalinclude, so this wrapping seems unneeded.
PostLoad listeners might initialize values for transient properties, so
the proxy should not skip initialization when using transient
properties.
Co-authored-by: Nicolas Grekas <nicolas.grekas@gmail.com>
The old proxy implementation of doctrine/common was triggered by public
methods rather than access to properties (making public properties
unsupported in entities), so tests could use public instance properties
to track the state of postLoad lifecycle callbacks without triggering
the proxy initialization when reading that state (which then changes the
state of triggering the postLoad callback).
As the new proxy implementation hooks into properties instead, the tests
now use a static method (ensuring it is reset properly before loading
the instance for which we care about the tracking) instead of an
instance property.
This fixes that using a `Criteria` with an `IN` or `NIN` expression on a to-many collection currently leads to an SQL error (#6173). The `ManyToMany` persister needs to know about the slightly different SQL syntax for `[NOT] IN ()`.
In the case of `[NOT] IN` expressions, the value will be an array, which also required me to change (I guess "fix") the parameter type handling. I have pulled the necessary code from the `BasicEntityPersister` and placed it as static helper methods in `PersisterHelper`.
This is somewhat inspired by #11516, which aims at fixing #11481: By re-using the parameter type handling code, it also fixes using backed enums in `EQ`, `IN` and `NIN` expressions within `Criteria` when `matching()` on one-to-many and many-to-many collections.
If scheduleForInsert was called in prePersist hook already, then persistNew need to check this case first, otherwise a ORMInvalidArgumentException will be thrown
This fixes a bug that arises when using Pagination and an entity relation is mapped with fetch-mode EAGER but setFetchMode LAZY (or anything that is not EAGER) has been used on the query. If the query use WITH condition, an exception is incorrectly raised (Associations with fetch-mode=EAGER may not be using WITH conditions).
The class LimitSubqueryOutputWalker clones the query, but not its parameters and hints, so the generated subquery does not know that fetch-mode has been overridden.
Fixes#11741
The bug related (#11694) and fixed mapping of sql column alias to field in entity (#11783) and
invalidate cache [cache/persisted/entity|cache/persisted/collection] when sql filter changes
* 2.20.x:
Introduce testNotListedValueInEnumArray
Fix documentation for JoinColumn nullable (#11798)
Ignore deprecations from doctrine/common
Fix fields of transient classes being considered duplicate with `reportFieldsWhereDeclared`
just one bracket (...) gives
Exception : [Doctrine\ORM\Query\QueryException] [Syntax Error] line 0, col xx: Error: Expected Doctrine\ORM\Query\Lexer::T_IDENTIFIER, got '('
When there are no conflicts between branches, we create pull requests
where the head branch is a branch on the origin repository. That branch
points to a commit that should already have coverage information
provided by the build that happens after merging a regular pull request.
The thing is, coverage information provided by builds that happen before
merging a pull request are associated with the commit of the head
repository. This means that when merging up 1.2 into 1.3, the build
produces coverage information that is the result of a merge between 1.2
and 1.3, and associates it with 1.2, although it is run on with a
codebase that is much closer to 1.3 (and is in fact supposed to become
1.3 after the merge).
This means that when we create a merge up PR from 1.2 to anything else,
the coverage information is going to be wrong until a PR targeting 1.2
gets merged.
I do not think we need coverage about conflictless merge up PRs more
than we need accurate numbers, so I propose we disable the upload for
those instead of, say, trying to associate them with the temporary merge
commit.
CachedPersisterContext::$selectJoinSql should be clear or regenerated when sqlFilter changed
The problem reproduce when in use fetch=EAGER and use additional sql filter on this property
I think it would be great to use literalinclude for big code snippets,
because our IDEs could warn us about issues, and it would be easily to
showcase our coding standard. Before we do that though, let us validate
that it renders as expected. I have picked a complex example where we
have a configuration block.
* Add a test covering the #11112 issue
* Add new OutputWalker and SqlFinalizer interfaces
* Add a SingleSelectSqlFinalizer that can take care of adding offset/limit as well as locking mode statements to a given SQL query.
Add a FinalizedSelectExecutor that executes given, finalized SQL statements.
* In SqlWalker, split SQL query generation into the two parts that shall happen before and after the finalization phase.
Move the part that generates "pre-finalization" SQL into a dedicated method. Use a side channel in SingleSelectSqlFinalizer to access the "finalization" logic and avoid duplication.
* Fix CS violations
* Skip the GH11112 test while applying refactorings
* Avoid a Psalm complaint due to invalid (?) docblock syntax
* Establish alternate code path - queries can obtain the sql executor through the finalizer, parser knows about output walkers yielding finalizers
* Remove a possibly premature comment
* Re-enable the #11112 test
* Fix CS
* Make RootTypeWalker inherit from SqlOutputWalker so it becomes finalizer-aware
* Update QueryCacheTest, since first/max results no longer need extra cache entries
* Fix ParserResultSerializationTest by forcing the parser to produce a ParserResult of the old kind (with the executor already constructed)
* Fix WhereInWalkerTest
* Update lib/Doctrine/ORM/Query/Exec/PreparedExecutorFinalizer.php
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
* Fix tests
* Fix a Psalm complaint
* Fix a test
* Fix CS
* Make the NullSqlWalker an instance of SqlOutputWalker
* Avoid multiple cache entries caused by LimitSubqueryOutputWalker
* Fix Psalm complaints
* Fix static analysis complaints
* Remove experimental code that I committed accidentally
* Remove unnecessary baseline entry
* Make AddUnknownQueryComponentWalker subclass SqlOutputWalker
That way, we have no remaining classes in the codebase subclassing SqlWalker but not SqlOutputWalker
* Use more expressive exception classes
* Add a deprecation message
* Move SqlExecutor creation to ParserResult, to minimize public methods available on it
* Avoid keeping the SqlExecutor in the Query, since it must be generated just in time (e. g. in case Query parameters change)
* Address PHPStan complaints
* Fix tests
* Small refactorings
* Add an upgrade notice
* Small refactorings
* Update the Psalm baseline
* Add a missing namespace import
* Update Psalm baseline
* Fix CS
* Fix Psalm baseline
---------
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
It maybe happen that the SQL COMMIT statement is successful, but then
something goes wrong. In that kind of case, you do not want to attempt a
rollback.
This was implemented in UnitOfWork::commit(), but for some reason not in
the similar EntityManager methods.
We use this method only from within one of our own test cases, and I don't see how it would be useful to anybody else outside – it has to be called on the `Parser` instance which exists internally in the `Query` only.
Deprecating and removing it in 3.x allows for a slight simplification in the `Parser` there, since we do no longer need the field (it can be a local variable).
People that might have experimented with property hooks while still
using ORM < 2.20.0 need to know that they need to remove their
experiment or upgrade to a version that explicitly supports them.
* 2.19.x:
Make nullable parameters explicit in generated entities (#11625)
Update attributes-reference.rst
Bump doctrine/.github from 5.0.1 to 5.1.0 (#11616)
Move orphan metadata to where it belongs
PHPStan 1.12 (#11585)
When adding the same lifecycle event callback to two or more lifecycle events, the generator will create a stub for each event resulting in fatal 'Cannot redeclare' errors. That is, only if the callback name contains uppercase letters.
If the source entity for an inverse (non-owning) 1-1 relationship is
identified by an association then the identifying association may not
be set when an inverse one-to-one association is resolved. This means
that no data is available in the entity to resolve the needed column
value for the join query.
The original entity data can be retrieved from the unit of work and
is used as a fallback to populate the query condition.
Fixes#11108
single-inheritence entity parent as targetEntity.
When using the parent entity for a single-inheritence table as the
targetEntity for a property, the discriminator value should be all
of the values in the discriminator map.
OneToManyPersister::deleteEntityCollection has been amended to
reflect this.
The Query class (used for DQL queries) takes care of using the value and
type as is when a type was specified for a parameter instead of going
through the default processing of values.
The NativeQuery class was missing the equivalent check, making the
custom type work only if the default processing of values does not
convert the value to a different one.
Remove redundant condition to check if target class contains foreign
identifier in order to allow fetching a null for relations with
composite keys, when part of the key value is null.
* 2.19.x:
Fix OneToManyPersister::deleteEntityCollection missing discriminator column/value. (GH-11500)
Skip joined entity creation for empty relation (#10889)
ci: maintained and stable mariadb version (11.4 current lts) (#11490)
fix(docs): use string value in `addAttribute`
Replace assertion with exception (#11489)
Use ramsey/composer-install in PHPBench workflow
update EntityManager#transactional to EntityManager#wrapInTransaction
Fix cloning entities
Consider usage of setFetchMode when checking for simultaneous usage of fetch-mode EAGER and WITH condition.
This fixes a bug that arises when an entity relation is mapped with
fetch-mode EAGER but setFetchMode LAZY (or anything that is not EAGER)
has been used on the query. If the query use WITH condition, an
exception is incorrectly raised (Associations with fetch-mode=EAGER may
not be using WITH conditions).
Fixes#11128
Co-Authored-By: Albert Prat <albert.prat@interactiu.cat>
If the entity gets reloaded from database before the deletions are
executed UnitOfWork needs to be able to return the original instance in
REMOVED state.
Changed capitalized column names to lowercase for consistency. Other occurances of column names mentioned as lowercase several times at this same page.
See https://github.com/doctrine/collections/issues/368 for the same
issue in doctrine/collections which has been fixed there.
The issue happens when using ->contains(). Running psalm emits
> InvalidArgument - Argument 1 of Doctrine\ORM\PersistentCollection::contains
> expects
> TMaybeContained:fn-doctrine\common\collections\readablecollection::contains
> as mixed, but … provided.
Solution: we should either not define @template TMaybeContained or
re-define the complete psalm docblock from ReadableCollection.
Repairing the docblock necessitates an update to the psalm baseline:
one "known issue" is no longer an issue and thus removed.
* Fix loading SchemaTool::getSchemaFromMetadata() uniqueConstraint without a name
Fixes a type miss-match exception when reading a UniqueConstraint defined on an Entity which doesn't have a predefined name.
* Fix deprecation on DBAL 3
---------
Co-authored-by: Alexander M. Turek <me@derrabus.de>
`getAssociationMappedByTargetField()` returns `null` when called with
the owning side of an association.
This is undocumented and wrong because the phpdoc advertises a string as
a return type.
Instead, callers should ensure they are calling that method with an
inverse side.
Closes#11250
Although this method is guaranteed to return either null or something
that can be used as a fully qualified class name, it never actually
checks that the class actually exists. Adding such a check breaks
several tests, including some that expect a exceptions at some later
points in the execution.
After 2.17 (some?) EAGER fetched OneToMany associations stopped working, if they have multiple join columns. Loads for these associations will trigger a `MessingPositionalParameter` exception "Positional parameter at index 1 does not have a bound value".
This test case should reproduce this issue, so it can be fixed.
* Remove trailing newlines
* Recommend safer way to disable logging
Resetting the middlewares on the configuration object will only work if
the connection object hasn't been built from that configuration object
yet. Instead, people should find the logger bound to the logging
middleware and disable it.
* Add TokenType class
Co-authored-by: Alexander M. Turek <me@derrabus.de>
* Deprecated Lexer constants in favour of TokenType
* Replace all Lexer::T_ occurrences with TokenType::T_
* Add upgrade note
* Fixed import Lexer => TokenType
* Fixed deprecation phpdoc
* Replaced int value with matching constant of TokenType
* Update src/Query/Lexer.php
---------
Co-authored-by: Alexander M. Turek <me@derrabus.de>
After commit 4e8e3ef30b when `\Doctrine\ORM\Query\SqlWalker` generates dicsriminator column condition SQL (method `\Doctrine\ORM\Query\SqlWalker::generateDiscriminatorColumnConditionSQL`) it adds an empty string to the list of possible values if the inheritance hierarchy contains a non-root abstract class.
When the discriminator column is implemented with a custom type in PostgreSQL (equivalent of Enum) the query fails because the type cannot have a value of an empty string. It boils down to the fact that `\Doctrine\ORM\Mapping\ClassMetadataInfo::$subClasses` contains an abstract class and in its Metadata the value of `\Doctrine\ORM\Mapping\ClassMetadataInfo::$discriminatorValue` is `null`.
#### Previous behavior
In version 2.14.1 `\Doctrine\ORM\Mapping\ClassMetadataInfo::$subClasses` does not contain an abstract class.
Fixes#11199, fixes#11177, fixes#10846.
---------
Co-authored-by: Michael Skvortsov <michael.skvortsov@eleving.com>
Co-authored-by: Matthias Pigulla <mp@webfactory.de>
The "any" tags inside the definition for mapped superclasses and
embeddables duplicate what is already done for entities.
The other removed "any" tags are also redundant, as they duplicate
what's already done inside the grandparent "choice" tag.
Starting with version libxml 2.12, such redundant tags cause errors
about the content model not being "determinist".
Fixes#11117
When using `AttributeOverride` to override mapping information inherited from a parent class (a mapped superclass), make sure to keep information about where the field was originally declared.
This is important for `private` fields: Without the correct `declared` information, it will lead to errors when cached mapping information is loaded, reflection wakes up and looks for the private field in the wrong class.
#10927 reported that #10455 broke the way how the default `@SequenceGeneratorDefinition` is created and inherited by subclasses for ID columns using `@GeneratedValue(strategy="SEQUENCE")`.
First, I had to understand how `@SequenceGeneratorDefinition` has been handled before #10455 when entity inheritance comes into play:
* Entity and mapped superclasses inherit the ID generator type (as given by `@GeneratedValue`) from their parent classes
* `@SequenceGeneratorDefinition`, however, is not generally inherited
* ... instead, a default sequence generator definition is created for every class when no explicit configuration is given. In this case, sequence names are based on the current class' table name.
* Once a root entity has been identified, all subclasses inherit its sequence generator definition unchanged.
#### Why did #10455 break this?
When I implemented #10455, I was mislead by two tests `BasicInheritanceMappingTest::testGeneratedValueFromMappedSuperclass` and `BasicInheritanceMappingTest::testMultipleMappedSuperclasses`.
These tests check the sequence generator definition that is inherited by an entity class from a mapped superclass, either directly or through an additional (intermediate) mapped superclass.
The tests expect the sequence generator definition on the entity _to be the same_ as on the base mapped superclass.
The reason why the tests worked before was the quirky behaviour of the annotation and attribute drivers that #10455 was aiming at: The drivers did not report the `@SequenceGeneratorDefinition` on the base mapped superclass where it was actually defined. Instead, they reported this `@SequenceGeneratorDefinition` for the entity class only.
This means the inheritance rules stated above did not take effect, since the ID field with the sequence generator was virtually pushed down to the entity class.
In #10455, I did not realize that these failing tests had to do with the quirky and changed mapping driver behaviour. Instead, I tried to "fix" the inheritance rules by passing along the sequence generator definition unchanged once the ID column had been defined.
#### Consequences of the change suggested here
This PR reverts the changes made to `@SequenceGeneratorDefinition` inheritance behaviour that were done in #10455.
This means that with the new "report fields where declared" driver mode (which is active in our functional tests) we can not expect the sequence generator definition to be inherited from mapped superclasses. The two test cases from `BasicInheritanceMappingTest` are removed.
I will leave a notice in #10455 to indicate that the new driver mode also affects sequence generator definitions.
The `GH10927Test` test case validates the sequence names generated in a few cases. In fact, I wrote this test against the `2.15.x` branch to make sure we get results that are consistent with the previous behaviour.
This also means `@SequenceGeneratorDefinition` on mapped superclasses is pointless: The mapped superclass does not make use of the definition itself (it has no table), and the setting is never inherited to child classes.
Fixes#10927. There is another implementation with slightly different inheritance semantics in #11052, in case the fix is not good enough and we'd need to review the topic later on.
In order to resolve#10348, some changes were included in #10547 to improve the computed _delete_ order for entities.
One assumption was that foreign key references with `ON DELETE SET NULL` or `... CASCADE` need not need to be taken into consideration when planning the deletion order, since the RDBMS would unset or cascade-delete such associations by itself when necessary. Only associations that do _not_ use RDBMS-level cascade handling would be sequenced, to make sure the referring entity is deleted before the referred-to one.
This assumption is wrong for `ON DELETE CASCADE`. The following examples give reasons why we need to also consider such associations, and in addition, we need to be able to deal with cycles formed by them.
In the following diagrams, `odc` means `ON DELETE CASCADE`, and `ref` is a regular foreign key with no extra `ON DELETE` semantics.
```mermaid
graph LR;
C-->|ref| B;
B-->|odc| A;
```
In this example, C must be removed before B and A. If we ignore the B->A dependency in the delete order computation, the result may not to be correct. ACB is not a working solution.
```mermaid
graph LR;
A-->|odc| B;
B-->|odc| A;
C-->|ref| B;
```
This is the situation in #10912. We have to deal with a cycle in the graph. C must be removed before A as well as B. If we ignore the B->A dependency (e.g. because we set it to "optional" to get away with the cycle), we might end up with an incorrect order ACB.
```mermaid
graph LR;
A-->|odc| B;
B-->|odc| A;
A-->|ref| C;
C-->|ref| B;
```
This example has no possible remove order. But, if we treat `odc` edges as optional, A -> C -> B would wrongly be deemed suitable.
```mermaid
graph LR;
A-->|ref| B;
B-->|odc| C;
C-->|odc| B;
D-->|ref| C;
```
Here, we must first remove A and D in any order; then, B and C in any order. If we treat one of the `odc` edges as optional, we might find the invalid solutions ABDC or DCAB.
#### Solution implemented in this PR
First, build a graph with a node for every to-be-removed entity, and edges for `ON DELETE CASCADE` associations between those entities. Then, use [Tarjan's algorithm](https://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm) to find strongly connected components (SCCs) in this graph. The significance of SCCs is that whenever we remove one of the entities in a SCC from the database (no matter which one), the DBMS will immediately remove _all_ the other entities of that group as well.
For every SCC, pick one (arbitrary) entity from the group to represent all entities of that group.
Then, build a second graph. Again we have nodes for all entities that are to be removed. This time, we insert edges for all regular (foreign key) associations and those with `ON DELETE CASCADE`. `ON DELETE SET NULL` can be left out. The edges are not added between the entities themselves, but between the entities representing the respective SCCs.
Also, for all non-trivial SCCs (those containing more than a single entity), add dependency edges to indicate that all entities of the SCC shall be processed _after_ the entity representing the group. This is to make sure we do not remove a SCC inadvertedly by removing one of its entities too early.
Run a topological sort on the second graph to get the actual delete order. Cycles in this second graph are a problem, there is no delete order.
Fixes#10912.
It will make fuzzy matchers more efficient, and configuration files more readable.
- lib/Doctrine/ORM becomes just src
- tests/Doctrine/ becomes just tests
Spotted while trying to merge https://github.com/doctrine/orm/pull/11076
(among other things) up into 3.0.x. On that branch, it is no longer
possible for an entity to extend another entity without specifying an
inheritance mapping type.
I think the goal of that inheritance was just to reuse the identifier
anyway, so let's just duplicate the identifier declaration instead.
This PR changes a detail in the commit order computation for depended-upon entities.
We have a parent-child relationship between two entity classes. The association is parent one-to-many children, with the child entities containing the (owning side) back-reference.
Cascade-persist is not used, so all entities have to be passed to `EntityManager::persist()`.
Before v2.16.0, two child entities C1 and C2 will be inserted in the same order in which they are passed to `persist()`, and that is regardless of whether the parent entity was passed to `persist()` before or after the child entities.
As of v2.16.0, passing the parent entity to `persist()` _after_ the child entities will lead to an insert order that is _reversed_ compared to the order of `persist()` calls.
This PR makes the order consistent in both cases, as it was before v2.16.0.
#### Cause
When the parent is passed to `persist()` after the children, commit order computation has to re-arrange the entities. The parent must be inserted first since it is referred to by the children.
The implementation of the topological sort from #10547 processed entities in reverse `persist()` order and unshifted finished nodes to an array to obtain the final result. That leads to dependencies (parent → before c1, parent → before c2) showing up in the result in the reverse order of which they were added.
This PR changes the topological sort to produce a result in the opposite order ("all edges pointing left"), which helps to avoid the duplicate array order reversal.
#### Discussion
* This PR _does not_ change semantics of the `persist()` so that entities would (under all ciscumstances) be inserted in the order of `persist()` calls.
* It fixes an unnecessary inconsistency between versions before 2.16.0 and after. In particular, it may be surprising that the insert order for the child entities depends on whether another referred-to entity (the parent) was added before or after them.
* _Both_ orders (c1 before or after c2) are technically and logically correct with regard to the agreement that `commit()` is free to arrange entities in a way that allows for efficient insertion into the database.
Fixes#11058.
Property names as returned by a cast to array are mangled, and that
mangling is not documented. Returning unprefixed produces the same
result, and is more likely to be supported by external tools relying on
the documented possible return values of __sleep.
For instance symfony/var-exporter does not support mangled names, which
leads to issues when caching query parsing results in Symfony
applications.
When unserializing from a cache entry in the previous format, the
sqlStatements need to be copied from the legacy property to the new
property before the reference is created.
The idea here is that instead of having a backward compatibility layer
in the next major branch, we can have a forward compatibility layer in
this branch.
When transforming these phpdoc types into native types, things break
down. They are correct according to the EBNF, but in practice, there are
so-called phase 2 optimizations that allow using ConditionalPrimary,
ConditionalFactor and ConditionalTerm instances in places where
ConditionalExpression is used.
doctrine/common has been split in several packages. A lot of what was
true about doctrine/common is true about doctrine/persistence today, so
let us simply reuse the existing paragraphs and mention persistence
instead of common.
This reduces our dependency to this shared library that now holds very
little code we use.
The class has not been copied verbatim:
- Unused parameters and methods have been removed.
- The class is final and internal.
- Coding standards have been enforced, including enabling strict_types,
which lead to casting a variable to string before feeding it to
explode().
- A bug found by static analysis has been addressed, where an INI
setting obtained with ini_get() was compared with true, which is never
returned by that function.
- Tests are improved to run on all PHP versions
What was optimal 10 years ago no longer is, and things might change in
the future. Using AUTO is still the best solution in most cases, and it
should be easy to make it mean something else when it is not.
* 2.16.x:
PHPStan 1.10.35, Psalm 5.15.0 (#10958)
docs: in text, refer to attributes when talking about metadata (#10956)
Fix bullet list layout (#10951)
docs[query-builder]: fix rendering of `Doctrine\DBAL\ParameterType::*` (#10945)
tests[ORMSetupTest]: testCacheNamespaceShouldBeGeneratedForApcu requires enabled apc (#10940)
docs: use modern named arguments syntax
Ignore "Unknown directive" error
Use a stable release
Remove output directory argument
tutorials[getting-started]: example fix bug id type definition
Verify UnitOfWork::HINT_DEFEREAGERLOAD exists and is true
This avoid situations where you might map a decimal to a float, when it
should really be mapped to a string. This is a big pitfall because in
that situation, the ORM will consider the field changed every time.
We have a lot of errors about "Unknown directive" that we should make
known when implementing guides for Doctrine, but cannot address by
modifying the docs.
The unknown directives are:
- configuration-block
- toc
- tocheader
- sectionauthor
* 2.16.x:
Use required classes for Lifecycle Callback examples (#10916)
Add space before backquote (#10918)
Add back throws annotation to getSingleScalarResult (#10907)
Fix link on known issues docs (#10904)
According to the RST docs,
> [inline markup] it must be separated from surrounding text by non-word
> characters. Use a backslash escaped space to work around that: thisis\ *one*\ word.
Because we were missing a space before backquotes here, the links were
not rendered. Escaping the space allow not to actually produce a space
in the output.
See https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#inline-markup
The changes from #10547, which landed in 2.16.0, cause problems for users calling `EntityManager::flush()` from within `postPersist` event listeners.
* When `UnitOfWork::commit()` is re-entered, the "inner" (reentrant) call will start working through all changesets. Eventually, it finishes with all insertions being performed and `UoW::$entityInsertions` being empty. After return, the entity insertion order, an array computed at the beginning of `UnitOfWork::executeInserts()`, still contains entities that now have been processed already. This leads to a strange-looking SQL error where the number of parameters used does not match the number of parameters bound. This has been reported as #10869.
* The fixes made to the commit order computation may lead to a different entity insertion order than previously. `postPersist` listener code may be affected by this when accessing generated IDs for other entities than the one the event has been dispatched for. This ID may not yet be available when the insertion order is different from the one that was used before 2.16. This has been mentioned in https://github.com/doctrine/orm/pull/10906#issuecomment-1682417987.
This PR suggests to address both issues by dispatching the `postPersist` event only after _all_ new entities have their rows inserted into the database. Likewise, dispatch `postRemove` only after _all_ deletions have been executed.
This solves the first issue because the sequence of insertions or deletions has been processed completely _before_ we start calling event listeners. This way, potential changes made by listeners will no longer be relevant.
Regarding the second issue, I think deferring `postPersist` a bit until _all_ entities have been inserted does not violate any promises given, hence is not a BC break. In 2.15, this event was raised after all insertions _for a particular class_ had been processed - so, it was never an "immediate" event for every single entity. #10547 moved the event handling to directly after every single insertion. Now, this PR moves it back a bit to after _all_ insertions.
Fix regression introduced in #10870
`$result = $this->execute(null, $hydrationMode);` in `getSingleResult` can still throw NoResultException exception.
#10880 reports a case where the changes from #10785 cause entity updates to be missed.
Upon closer inspection, this change seems to be causing it:
https://github.com/doctrine/orm/pull/10785/files#diff-55a900494fc8033ab498c53929716caf0aa39d6bdd7058e7d256787a24412ee4L2990-L3003
The code was changed to use `registerManaged()` instead, which basically does the same things, but (since #10785) also includes an additional check against duplicate entity instances.
But, one detail slipped through tests and reviews: `registerManaged()` also updates `\Doctrine\ORM\UnitOfWork::$originalEntityData`, which is used to compute entity changesets. An empty array `[]` was passed for $data here.
This will make the changeset computation assume that a partial object was loaded and effectively ignore all field updates here:
a616914887/lib/Doctrine/ORM/UnitOfWork.php (L762-L764)
I think that, effectively, it is sufficient to call `registerManaged()` only in the two cases where a proxy was created.
Calling `registerManaged()` with `[]` as data for a proxy object is consistent with e. g. `\Doctrine\ORM\EntityManager::getReference()`.
In the case that a full entity has to be loaded, we need not call `registerManaged()` at all, since that will already happen inside `EntityManager::find()` (or, more specifically, `UnitOfWork::createEntity()` called inside it).
Note that the test case has to make some provisions so that we actually reach this case:
* Load an entity that uses `fetch="EAGER"` on a to-one association
* That association being against a class that uses inheritance (why's that?)
* 2.16.x:
Turn identity map collisions from exception to deprecation notice (#10878)
Add possibility to set reportFieldsWhereDeclared to true in ORMSetup (#10865)
Fix UnitOfWork->originalEntityData is missing not-modified collections after computeChangeSet (#9301)
Add an UPGRADE notice about the potential changes in commit order (#10866)
Update branch metadata (#10862)
In #10785, a check was added that prevents entity instances from getting into the identity map when another object for the same ID is already being tracked.
This caused regressions for users that work with application-provided IDs and expect this condition to fail with `UniqueConstraintViolationExceptions` when flushing to the database.
Thus, this PR turns the exception into a deprecation notice. Users can opt-in to the new behavior. In 3.0, the exception will be used.
Implements #10871.
* Fix original data incomplete after flush
* Apply suggestions from code review
Co-authored-by: Alexander M. Turek <me@derrabus.de>
---------
Co-authored-by: Alexander M. Turek <me@derrabus.de>
This excludes such associations from the commit order computation, since the foreign key constraint will be satisfied when inserting the row.
See https://github.com/doctrine/orm/pull/10735/ for more details about this edge case.
1. Inherit ClassMetadataInfo::requiresFetchAfterChange flag from root entity when process parent columns mapping (see ClassMetadataInfo::addInheritedFieldMapping(), it uses same condition as ClassMetadataInfo::mapField()) so JoinedSubclassPersister::assignDefaultVersionAndUpsertableValues() to be called in JoinedSubclassPersister::executeInserts().
2. Override JoinedSubclassPersister::fetchVersionAndNotUpsertableValues() to fetch all parent tables (see $this->getJoinSql() call) generated columns. So make protected BasicEntityPersister::identifierFlattener stateless service (use it flattenIdentifier() method) and BasicEntityPersister::extractIdentifierTypes() (to avoid copy-paste).
3. JoinedSubclassPersister::fetchVersionAndNotUpsertableValues() doesnt check empty $columnNames because it would be an error if ClassMetadataInfo::requiresFetchAfterChange is true while no generated columns in inheritance hierarchy.
4. Initialize JoinedInheritanceRoot not-nullable string properties with insertable=false attribute to avoid attempt to insert default null data which cause error:
PDOException: SQLSTATE[23502]: Not null violation: 7 ERROR: null value in column "rootwritablecontent" of relation "joined_inheritance_root" violates not-null constraint
DETAIL: Failing row contains (5, null, dbDefault, dbDefault, , nonUpdatable).
while $rootTableStmt->executeStatement() because JoinedSubclassPersister::getInsertColumnList() have no $insertData (prepared later) to decide is generated column provided by client code or not (so need to skip column)
1. Postgres gives error when insert root entity ($rootTableStmt->executeStatement()) in JoinedSubclassPersister::executeInserts():
PDOException: SQLSTATE[08P01]: <<Unknown error>>: 7 ERROR: bind message supplies 4 parameters, but prepared statement "" requires 6
so exclude notInsertable columns from JoinedSubclassPersister::getInsertColumnList() like it done in parent::prepareInsertData() which call BasicEntityPersister::prepareUpdateData(isInsert: true) where we have condition:
if ($isInsert && isset($fieldMapping['notInsertable']))
2. Try to get generated (notInsertable|notUpdatable) column value on flush() with JoinedSubclassPersister::executeInserts() also fails:
Unexpected empty result for database query.
because method it calls $this->assignDefaultVersionAndUpsertableValues() after insert root entity row, while generated columns in child-entity table, so move call just after insert child row
3. Use option['default'] = 'dbDefault' in functional test entities, to emulate generated value on insert, but declare as generated = 'ALWAYS' for tests purpose (correctness of JoinedSubclassPersister::fetchVersionAndNotUpsertableValues() sql-query)
4. Use JoinedInheritanceRoot::rootField to skip JoinedSubclassPersister::update() optimization for empty changeset in updatable:false columns tests
In 7fa3e6ec7c, a global search and replace
was used for http and https.
This broke the documentation examples in that as soon as you turn on XSD
validation, it will fail because the namespace in the XML file does not
match the ones defined in the XSD file, which do not exhibit the https.
Note that this is not a security concern, because these URIs are
not meant to be actually resolved, but to serve as a unique identifier
for the namespace in which we define our elements.
This change improves scheduling of extra updates in the `BasicEntityPersister`.
Extra updates can be avoided when
* the referred-to entity has already been inserted during the current insert batch/transaction
* we have a self-referencing entity with application-provided ID values (the `NONE` generator strategy).
As a corollary, with this change applications that provide their own IDs can define self-referencing associations as not NULLable.
I am considering this a bugfix since the ORM previously executed additional queries that were not strictly necessary, and that required users to work with NULLable columns where conceptually a non-NULLable column would be valid and more expressive.
One caveat, though:
In the absence of entity-level commit ordering (#10547), it is not guaranteed that entities with self-references (at the class level) will be inserted in a suitable order. The order depends on the sequence in which the entities were added with `persist()`.
Fixes#7877, closes#7882.
Co-authored-by: Sylvain Fabre <sylvain.fabre@assoconnect.com>
The way we have our docs, the sidebar is a separate document and as
such, needs a title. Let us prepend a dummy title until the guides-cli
provides a way to ignore that particular error.
* 2.15.x: (23 commits)
Fix cloning entities when using lazy-ghost proxies
Fixes recomputation of single entity change set when entity contains enum attributes. Due to the fact that originalEntityData contains enum objects and ReflectionEnumProperty::getValue() returns value of enum, comparison of values are always falsy, resulting to update columns value even though it has not changes.
Fix code style issues
Use absolute references
Avoid colon followed by double colon
Use correct syntax for references
Fix invalid reference syntax
Use rst syntax
Use internal link
Escape pipes
Introduce new workflow to test docs
Remove lone dash (#10812)
Treat id field proprites same as regular field
Move three "Ticket/"-style tests to the right namespace
Follow recommendation about multiline type
Fix unserialize() errors when running tests on PHP 8.3 (#10803)
Explain `EntityManager::getReference()` peculiarities (#10800)
Upgrade to Psalm 5.13
test: assert `postLoad` has data first
distinct() updates QueryBuilder state correctly
...
Due to the fact that originalEntityData contains enum objects and
ReflectionEnumProperty::getValue() returns value of enum,
comparison of values are always falsy, resulting to update
columns value even though it has not changes.
According to the Sphinx docs, when in reference/architecture.rst, a
reference to reference/inheritance-mapping would resolve to
reference/reference/inheritance-mapping.rst, because it is relative to
the current document
There are a few requests (#5742, #5368, #5109, #6776) that ask to change the order of operations in the UnitOfWork to perform "deletes before inserts", or where such a switch appears to solve a reported problem.
I don't want to say that this is not doable. But this PR at least adds two tricky examples where INSERTs need to be done before an UPDATE can refer to new database rows; and where the UPDATE needs to happen to release foreign key references to other entities before those can be DELETEd.
So, at least as long as all operations of a certain type are to be executed in blocks, this example allows no other order of operations than the current one.
While trying to understand #3037, I found that it may happen that we have more entries in `\Doctrine\ORM\UnitOfWork::$entityIdentifiers` than in `\Doctrine\ORM\UnitOfWork::$identityMap`.
The former is a mapping from `spl_object_id` values to ID hashes, the latter an array first of entity class names and then from ID hash to entity object instances.
(Basically, "ID hash" is a concatenation of all field values making up the `@Id` for a given entity.)
This means that at some point, we must have _different_ objects representing the same entity, or at least over time different objects are used for the same entity without the UoW properly updating its `::$entityIdentifiers` structure.
I don't think it makes sense to overwrite an entity in the identity map, since that means a currently `MANAGED` entity is replaced with something else.
If it makes sense at all to _replace_ an entity, that should happen through dedicated management methods to first detach the old entity before persisting, merging or otherwise adding the new one. This way we could make sure the internal structures remain consistent.
Apparently, there is consensus about multiline types between:
- PHPStan
- Psalm
- Slevomat Coding Standard
See https://github.com/slevomat/coding-standard/issues/1586#issuecomment-1610195706
Using parenthesis is less ambiguous, it makes it clear to the parser
where the type begins and where it ends.
The change has a positive impact on the Psalm baseline, showing
that psalm-return annotation was not really understood previously.
A lot of our tests mention it, but I do not think it is important to the
test. Maybe it was a way to have more efficient tests? Most times,
removing the hint does not affect the test outcome.
* Explain `EntityManager::getReference()` peculiarities
As one takeaway from https://github.com/doctrine/orm/issues/3037#issuecomment-1605657003 and #843, we should look into better explaining the `EntityManager::getReference()` method, it’s semantics, caveats and potential responsibilities placed on the user.
This PR tries to do that, so it fixes#10797.
* Update docs/en/reference/advanced-configuration.rst
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
---------
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
Previously calling distinct() when the QueryBuilder was in clean state would cause subsequent getDQL() calls to ignore the distinct queryPart
Fixes#10784
This PR tries to improve the situation/problem explained in #3037:
Under certain conditions – there may be multiple and not all are known/well-understood – we may get inconsistencies between the `\Doctrine\ORM\UnitOfWork::$entityIdentifiers` and `\Doctrine\ORM\UnitOfWork::$identityMap` arrays.
Since the `::$identityMap` is a plain array holding object references, objects contained in it cannot be garbage-collected.
`::$entityIdentifiers`, however, is indexed by `spl_object_id` values. When those objects are destructed and/or garbage-collected, the OID may be reused and reassigned to other objects later on.
When the OID re-assignment happens to be for another entity, the UoW may assume incorrect entity states and, for example, miss INSERT or UPDATE operations.
One cause for such inconsistencies is _replacing_ identity map entries with other object instances: This makes it possible that the old object becomes GC'd, while its OID is not cleaned up. Since that is not a use case we need to support (IMHO), #10785 is about adding a safeguard against it.
In this test shown here, the `merge()` operation is currently too eager in creating a proxy object for another referred-to entity. This proxy represents an entity already present in the identity map at that time, potentially leading to this problem later on.
Add tests for entity insertion and deletion that require the commit order calculation to happen on the entity level. This demonstrates the necessity for the changes in #10547.
This PR contains two tests with carefully constructed entity relationships, where we have a non-nullable `parent` foreign key relationships between entities stored in the same table.
Class diagram:
```mermaid
classDiagram
direction LR
class A
class B
A --> B : parent
B --|> A
```
Object graph:
```mermaid
graph LR;
b1 --> b2;
b2 --> a;
b3 --> b2;
```
#### Situation before #10547
The commit order is computed by looking at the associations at the _table_ (= _class_) level. Once the ordering of _tables_ has been found, entities being mapped to the same table will be processed in the order they were given to `persist()` or `remove()`.
That means only a particular ordering of `persist()` or `remove()` calls (see comment in the test) works:
For inserts, the order must be `$a, $b2, $b1, $b3` (or `... $b3, $b1`), for deletions `$b1, $b3, $b2, $a`.
#### Situation with entity-level commit order computation (as in #10547)
The ORM computes the commit order by considering associations at the _entity_ level. It will be able to find a working order by itself.
This refactoring does two things:
* We can avoid collecting the post insert IDs in a cumbersome array structure that will be returned by the EntityPersisters and processed by the UoW right after. Instead, use a more expressive API: Make the EntityPersisters tell the UoW about the IDs immediately.
* IDs will be available in inserted entities a tad sooner. That may help to resolve#10735, where we can use the IDs to skip extra updates.
When collection updates/join table cleanups do not happen through specialized Entity-/CollectionPersister methods but instead as "plain" updates, we may issue a lot more queries than expected.
In my case a custom doctrine type of Uuid object is converted to string by simply casting it, resulting in a hex DELETE FROM x WHERE id = ? query,
whilst it should've been converted along the way to it's binary representation. This leads to no deletions being made at all as you would expect making use of doctrine custom type's as an identifier.
This commit fixes usage of ramsey/uuid or symfony/uid as custom id types when making use of orphan removal.
This is the result of the contradiction between the phpdoc
(ClassMetadata), and the condition, which guarantees $metadata is not a
ClassMetadata. Relaxing the phpdoc leads to other phpstan issues, about
properties that exist in ClassMetadata but not in
PersistentClassMetadata. The right way to fix this would be to switch
from a deprecation to an exception, but that is not the path we have
taken, and all this will disappear in 3.0.x anyway, so let's not bother.
Add tests for entity insertion and deletion that require writes to different tables in an interleaved fashion, and that have to re-visit a particular table.
#### Background
In #10531, I've given an example where it is necessary to compute the commit order on the entity (instead of table) level.
Taking a closer look at the UoW to see how this could be achieved, I noticed that the current, table-level commit order manifests itself also in the API between the UoW and `EntityPersister`s.
#### Current situation
The UoW computes the commit order on the table level. All entity insertions for a particular table are passed through `EntityPersister::addInsert()` and finally written through `EntityPersister::executeInserts()`.
#### Suggested change
The test in this PR contains a carefully constructed set of four entities. Two of them are of the same class (are written to the same table), but require other entities to be processed first.
In order to be able to insert this set of entities, the ORM must be able to perform inserts for a given table repeatedly, interleaved with writing other entities to their respective tables.
This is part of the series of issues fixed by #10547. In particular, the changes from #10566 were relevant.
See #10348 for the bug description.
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
This is the third step to break https://github.com/doctrine/orm/pull/10547 into smaller PRs suitable for reviewing. It uses the new topological sort implementation from #10592 and the refactoring from #10651 to compute the UoW's commit order for entity insertions not on the entity class level, but for single entities and their actual dependencies instead.
#### Current situation
`UnitOfWork::getCommitOrder()` would compute the entity sequence on the class level with the following code:
70477d81e9/lib/Doctrine/ORM/UnitOfWork.php (L1310-L1325)
#### Suggested change
* Instead of considering the classes of all entities that need to be inserted, updated or deleted, consider the new (inserted) entities only. We only need to find a sequence in situations where there are foreign key relationships between two _new_ entities.
* In the dependency graph, add edges for all to-one association target entities.
* Make edges "optional" when the association is nullable.
#### Test changes
I have not tried to fully understand the few changes necessary to fix the tests. My guess is that those are edge cases where the insert order changed and we need to consider this during clean-up.
Keep in mind that many of the functional tests we have assume that entities have IDs assigned in the order that they were added to the EntityManager. That does not change – so the order of entities is generally stable, equal to the previous implementation.
When computing the commit order for entity removals, we have to look out for `@ORM\JoinColumn(onDelete="SET NULL")` to find places where cyclic associations can be broken.
#### Background
The UoW computes a "commit order" to find the sequence in which tables shall be processed when inserting entities into the database or performing delete operations.
For the insert case, the ORM is able to schedule _extra updates_ that will be performed after all entities have been inserted. Associations which are configured as `@ORM\JoinColumn(nullable=true, ...)` can be left as `NULL` in the database when performing the initial `INSERT` statements, and will be updated once all new entities have been written to the database. This can be used to break cyclic associations between entity instances.
For removals, the ORM does not currently implement up-front `UPDATE` statements to `NULL` out associations before `DELETE` statements are executed. That means when associations form a cycle, users have to configure `@ORM\JoinColumn(onDelete="SET NULL", ...)` on one of the associations involved. This transfers responsibility to the DBMS to break the cycle at that place.
_But_, we still have to perform the delete statements in an order that makes this happen early enough. This may be a _different_ order than the one required for the insert case. We can find it _only_ by looking at the `onDelete` behaviour. We must ignore the `nullable` property, which is irrelevant, since we do not even try to `NULL` anything.
#### Example
Assume three entity classes `A`, `B`, `C`. There are unidirectional one-to-one associations `A -> B`, `B -> C`, `C -> A`. All those associations are `nullable= true`.
Three entities `$a`, `$b`, `$c` are created from these respective classes and associations are set up.
All operations `cascade` at the ORM level. So we can test what happens when we start the operations at the three individual entities, but in the end, they will always involve all three of them.
_Any_ insert order will work, so the improvements necessary to solve #10531 or #10532 are not needed here. Since all associations are between different tables, the current table-level computation is good enough.
For the removal case, only the `A -> B` association has `onDelete="SET NULL"`. So, the only possible execution order is `$b`, `$c`, `$a`. We have to find that regardless of where we start the cascade operation.
The DBMS will set the `A -> B` association on `$a` to `NULL` when we remove `$b`. We can then remove `$c` since it is no longer being referred to, then `$a`.
#### Related cases
These cases ask for the ORM to perform the extra update before the delete by itself, without DBMS-level support:
* #5665
* #10548
mappedBy is never defined on an inverse relationship.
Bi-directional self-referencing should IMO still result in 2 separate
associations, on 2 separate fields of the same class, like mentor or
mentee.
This is the first chunk to break #10547 into smaller PRs suitable for reviewing. It adds a new topological sort implementation.
#### Background
Topological sort is an algorithm that sorts the vertices of a directed acyclic graph (DAG) in a linear order such that for every directed edge from vertex A to vertex B, vertex A comes before vertex B in the ordering. This ordering is called a topological order.
Ultimately (beyond the scope of this PR), in the ORM we'll need this to find an order in which we can insert new entities into the database. When one entity needs to refer to another one by means of a foreign key, the referred-to entity must be inserted before the referring entity. Deleting entities is similar.
A topological sorting can be obtained by running a depth first search (DFS) on the graph. The order in which the DFS finishes on the vertices is a topological order. The DFS is possible iif there are no cycles in the graph. When there are cycles, the DFS will find them.
For more information about topological sorting, as well as a description of an DFS-based topological sorting algorithm, see https://en.wikipedia.org/wiki/Topological_sorting.
#### Current situation
There is a DFS-based topological sorting implemented in the `CommitOrderCalculator`. This implementation has two kinks:
1. It does not detect cycles
When there is a cycle in the DAG that cannot be resolved, we need to know about it. Ultimately, this means we will not be able to insert entities into the database in any order that allows all foreign keys constraints to be satisfied.
If you look at `CommitOrderCalculator`, you'll see that there is no code dealing with this situation.
2. It has an obscure concept of edge "weights"
To me, it is not clear how those are supposed to work. The weights are related to whether a foreign key is nullable or not, but can (could) be arbitrary integers. An edge will be ignored if it has a higher (lower) weight than another, already processed edge... 🤷🏻?
#### Suggested change
In fact, when inserting entities into the database, we have two kinds of foreign key relationships: Those that are `nullable`, and those that are not.
Non-nullable foreign keys are hard requirements: Referred-to entities must be inserted first, no matter what. These are "non-optional" edges in the dependency graph.
Nullable foreign keys can be set to `NULL` when first inserting an entity, and then coming back and updating the foreign key value after the referred-to (related) entity has been inserted into the database. This is already implemented in `\Doctrine\ORM\UnitOfWork::scheduleExtraUpdate`, at the expense of performing one extra `UPDATE` query after all the `INSERT`s have been processed. These edges are "optional".
When finding a cycle that consists of non-optional edges only, treat it as a failure. We won't be able to insert entities with a circular dependency when all foreign keys are non-NULLable.
When a cycle contains at least one optional edge, we can use it to break the cycle: Use backtracking to go back to the point before traversing the last _optional_ edge. This omits the edge from the topological sort order, but will cost one extra UPDATE later on.
To make the transition easier, the new implementation is provided as a separate class, which is marked as `@internal`.
#### Outlook
Backtracking to the last optional edge is the simplest solution for now. In general, it might be better to find _another_ (optional) edge that would also break the cycle, if that edge is also part of _other_ cycles.
Remember, every optional edge skipped costs an extra UPDATE query later on. The underlying problem is known as the "minimum feedback arc set" problem, and is not easily/efficiently solvable. Thus, for the time being, picking the nearest optional edge seems to be reasonable.
This PR will make the annotations and attribute mapping drivers report mapping configuration for the classes where it is declared, instead of omitting it and reporting it for subclasses only. This is necessary to be able to catch mis-configurations in `ClassMetadataFactory`.
Fixes#10417, closes#10450, closes#10454.
#### ⚠️ Summary for users getting `MappingExceptions` with the new mode
When you set the `$reportFieldsWhereDeclared` constructor parameters to `true` for the AnnotationDriver and/or AttributesDriver and get `MappingExceptions`, you may be doing one of the following:
* Using `private` fields with the same name in different classes of an entity inheritance hierarchy (see #10450)
* Redeclaring/overwriting mapped properties inherited from mapped superclasses and/or other entities (see #10454)
As explained in these two PRs, the ORM cannot (or at least, was not designed to) support such configurations. Unfortunately, due to the old – now deprecated – driver behaviour, the misconfigurations could not be detected, and due to previously missing tests, this in turn was not noticed.
#### Current situation
The annotations mapping driver has the following condition to skip properties that are reported by the PHP reflection API:
69c7791ba2/lib/Doctrine/ORM/Mapping/Driver/AnnotationDriver.php (L345-L357)
This code has been there basically unchanged since the initial 2.0 release. The same condition can be found in the attribute driver, probably it has been copied when attributes were added.
I _think_ what the driver tries to do here is to deal with the fact that Reflection will also report `public`/`protected` properties inherited from parent classes. This is supported by the observation (see #5744) that e. g. YAML and XML drivers do not contain this logic.
The conditions are not precise enough for edge cases. They lead to some fields and configuration values not even being reported by the driver.
Only since the fields would be "discovered" again when reflecting on subclasses, they eventually end up in class metadata structures for the subclasses. In one case of inherited ID generator mappings, the `ClassMetadataFactory` would also rely on this behaviour.
Two potential bugs that can result from this are demonstrated in #10450 and #10454.
#### Suggested solution
In order to find a more reliable way of separating properties that are merely reported again in subclasses from those that are actual re-declarations, use the information already available in `ClassMetadata`. In particular, `declared` tells us in which non-transient class a "field" was first seen.
Make the mapping driver skip only those properties for which we already know that they have been declared in parent classes, and skip them only when the observed declaring class matches the expectation.
For all other properties, report them to `ClassMetadataFactory` and let that deal with consistency checking/error handling.
#10450 and #10454 are merged into this PR to show that they pass now.
#### Soft deprecation strategy
To avoid throwing new/surprising exceptions (even for misconfigurations) during a minor version upgrade, the new driver mode is opt-in.
Users will have to set the `$reportFieldsWhereDeclared` constructor parameters to `true` for the `AnnotationDriver` and/or `AttributesDriver`. Unless they do so, a deprecation warning will be raised.
In 3.0, the "new" mode will become the default. The constructor parameter can be deprecated (as of ORM 3.1, probably) and is a no-op.
We need to follow up in other places (DoctrineBundle, ... – what else?) to make this driver parameter an easy-to-change configuration setting.
In the code examples of the use of the ``Query#toIterable()`` method, an index ``$i`` is used to control the size of the current batch, which ends when the condition ``($i % $batchSize) === 0`` becomes true.
Because this condition is preceded by ``++$i`` and ``$i`` is initialized to 1, the first batch is actually of size ``$batchSize - 1`` instead of ``$batchSize``.
To solve this, ``$i`` should be initialized to 0 instead of 1.
Currently, the AttributeDriver ignores any join column attribute specified on a many to many relationship.
Let's copy code from the AnnotationDriver to fix that.
Fixes#9902
Following up on #10652:
#### Current situation
The implementation of `\Doctrine\ORM\Mapping\ClassMetadataInfo::_validateAndCompleteOneToOneMapping` will consider a field with a one-to-one association to be the owning side also when it configures `@JoinColumn` settings.
#### Suggested change
For a one to one association, a field should be the inverse side when it uses the `mappedBy` attribute, and be the owning side otherwise. The `JoinColumn` may be configured on the owning side only.
This PR adds a deprecation notice when `@JoinColumn` is used on the side of a one-to-one association where `mappedBy` occurs.
In 3.0, this will throw a `MappingException`.
This makes no sense because it describes a one-to-one where each table
references the other one. I do not think this is deliberately supported,
and it probably will not be supported at all in 3.0.
This is the second chunk to break #10547 into smaller PRs suitable for reviewing. It prepares the `UnitOfWork` to work with a commit order computed on the entity level, as opposed to a class-based ordering as before.
#### Background
#10531 and #10532 show that it is not always possible to run `UnitOfWork::commit()` with a commit order given in terms of entity _classes_. Instead it is necessary to process entities in an order computed on the _object_ level. That may include
* a particular order for multiple entities of the _same_ class
* a particular order for entities of _different_ classes, possibly even going back and forth (entity `$a1` of class `A`, then `$b` of class `B`, then `$a2` of class `A` – revisiting that class).
This PR is about preparing the `UnitOfWork` so that its methods will be able to perform inserts and deletions on that level of granularity. Subsequent PRs will deal with implementing that particular order computation.
#### Suggested change
Change the private `executeInserts` and `executeDeletions` methods so that they do not take a `ClassMetadata` identifying the class of entities that shall be processed, but pass them the list of entities directly.
The lists of entities are built in two dedicated methods. That happens basically as previously, by iterating over `$this->entityInsertions` or `$this->entityDeletions` and filtering those by class.
#### Potential (BC breaking?) changes that need review scrutiny
* `\Doctrine\ORM\Persisters\Entity\EntityPersister::addInsert()` was previously called multiple times, before the insert would be performed by a call to `\Doctrine\ORM\Persisters\Entity\EntityPersister::executeInserts()`. With the changes here, this batching effectively no longer takes place. `executeInserts()` will always see one entity/insert only, and there may be multiple `executeInserts()` calls during a single `UoW::commit()` phase.
* The caching in `\Doctrine\ORM\Cache\Persister\Entity\AbstractEntityPersister` previously would cache entities from the last executed insert batch only. Now it will collect entities across multiple batches. I don't know if that poses a problem.
* Overhead in `\Doctrine\ORM\Persisters\Entity\BasicEntityPersister::executeInserts` is incurred multiple times; that may, however, only be about SQL statement preparation and might be salvageable.
* The `postPersist` event previously was not dispatched before _all_ instances of a particular entity class had been written to the database. Now, it will be dispatched immediately after every single entity that has been inserted.
* Add test case for https://github.com/doctrine/orm/issues/7717
* Do not hide null equality checks in `SqlValueVisitor::walkComparison`
* Annotate `GH7717Parent::$children` type
When an entity have a backed enum as identifier, `UnitOfWork` tries to
cast to string when generating the hash of the id.
This fix calls `->value` when identifier is a `BackedEnum`.
Fixes#10471Fixes#10334
DiscriminatorColumnMapping is just a specialization of the array shape
that is right of the pipe: it has the same fields, except fewer fields
are nullable. The union of that is the same thing as the array shape.
The default allocationSize for @SequenceGenerator has actually been set to 1 ever since 434325e (back in 2010!)
This was done to remove confusion, but the docs were never updated to reflect the change
This puts the remarks that apply to both JTI/STI in a common section at the beginning, and tries to explain a bit more in detail why it is that way.
It was sparked off by the discussion in #10538.
* 2.14.x:
Ignore the cache dir of PHPUnit 10 (#10546)
Make data providers static (#10544)
Bump dev tools (#10541)
Mark SqlWalker methods as not deprecated (#10540)
docs: consistency order for docblock in association mapping (#10534)
Correct use of PHP attribute
fix typo in faq.rst (#10526)
fix: use executeStatement in SchemaTool (#10516)
Write a test in a more specific way
Put up a warning sign that mapping may not be inherited from transient classes (#10392)
Avoid unnecessary information in query hints to improve query cache hit ratio
phpstan treats implementations of deprecated methods of an interface as being deprecated themselves by default.
However, SqlWalker does not intend to deprecate all those methods that are deprecated in TreeWalker, as they are
moved down. Marking them as not deprecated will avoid reporting usages of deprecated APIs when implementing
custom DQL functions for instance.
Working on converting these array shapes to DTO allowed me to find every
signature where they are supposed to be used.
The Psalm baseline gets worse because it considers accessing an array
key differently depending on whether it is defined vaguely, as
array<string, mixed>, or precisely, as array{my-key?: string}.
This _seems_ to work, but...
To my understanding, that is only because `ReflectionClass::getProperties` will report `protected` properties also for subclasses, including DocBlocks etc. The mapping drivers are unable to tell where a field comes from, so they pick up the mapping and treat it as originating from the inheriting class.
If that is indeed outside of what the ORM was designed for (sombody please confirm?), then we should explicitly discourage it.
Yes, I have seen examples of this in the wild.
Allow to-many associations to be used on mapped superclasses when the owning (inverse) side does not refer back to the mapped superclass, thanks to `ResolveTargetEntityListener`.
#### Current situation
The [documentation states](https://www.doctrine-project.org/projects/doctrine-orm/en/latest/reference/inheritance-mapping.html):
> No database table will be created for a mapped superclass itself
> [...] persistent relationships defined by a mapped superclass must be unidirectional (with an owning side only). This means that One-To-Many associations are not possible on a mapped superclass at all.
That's a though limitation.
~Obviously~ ~apparently~ Probably the limitation comes from the fact that in a to-many association the "many" side has to hold a foreign key. Since the mapped superclass does not have a database table (it's not an entity), no such backreference can be established.
Currently, to-many associations trigger an exception as soon as they are seen on a mapped superclass:
d6c0031d44/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php (L459-L461)
#### `ResolveTargetEntityListener`
The `ResolveTargetEntityListener` can be used to substitute interface or class names in mapping configuration at runtime, during the metadata load phase.
When this gimmick is used to replace _all_ references to the mapped superclass with an entity class in time, it should be possible to have to-many associations on the inheriting entity classes.
#### Suggested solution
Instead of rejecting to-many associations on mapped superclasses right away, validate that at the end of the day (after the `loadClassMetadata` event has been processed) no association may target at a non-entity class. That includes mapped superclasses as well as transient classes.
#### Motivating example
Consider a library that comes with a `User` base class. This class is `abstract` and has to be subclassed/filled when the library is used.
By making this a mapped superclass, library users have the freedom to either have a simple user entity class or a user class hierarchy, but we do not impose any requirements on them. (NB we also don't want to have a root entity in the library, because that would have to declare the entire class hierarchy, including library users' classes.)
The actual user class to be used will be configured through the `ResolveTargetEntityListener`.
The library also includes a `SocialMediaAccount` entity. A `User` can have multiple of these accounts, and we want to be able to navigate the accounts from the user side.
To make the example even more fancy, there is a self-referencing association on the `User`: A `User` has been created by another user, and holds a collection of all other `User`s it created.
The test case contained in this PR contains this example and validates that all association mappings look just as if the final user class had been written as an entity directly, without the superclass.
#### Potential review talking points
- Am I missing other reasons why to-many is not feasible?
- We now reject association mappings with `targetEntity`s that are not entities; relevant BC break? (IMHO: no.)
#### Review tip
Review commit by commit, not all files at once. The last commit adds a lot of entity declarations that were previously missed in tests and now raised exceptions; that's a lot of clutter in the PR.
I've noticed that over time my query caches fill up with redundant queries, i. e. different cache entries for the DQL -> SQL translation that are exactly the same. For me, it's an issue because the cache entries fill up precious OPcache memory.
Further investigation revealed that the queries themselves do not differ, but only the query hints – that are part of the computed cache key – do.
In particular, only the value for the `WhereInWalker::HINT_PAGINATOR_ID_COUNT` query hint are different. Since `WhereInWalker` only needs to know _if_ there are matching IDs but not _how many_, we could avoid such cache misses by using just a boolean value as cache hint.
One could interpret the old description as if `Connection#transactional()` would not rollback the transaction. Also, the fact that the `EntityManager` gets closed in case of an exception was not mentioned.
* 2.14.x:
Remove calls to assertObjectHasAttribute() (#10502)
Remove calls to withConsecutive() (#10501)
Use recognized array key
Fix#9095 by re-applying #9096
Use linebreaks
"joinColumn" has no meaning for the static PHP driver. That's because it
performs no translation, unlike other drivers: we're using the
ClassMetadata API directly.
Before this change, changing the value of the "name" key did not break
the test suite. After this change, it does.
Make the `Paginator`-internal query (`... WHERE ... IN (id, id2,
id3...)`) cacheable in the query cache again.
When the Paginator creates the internal subquery that does the actual
result limiting, it has to take DBAL type conversions for the identifier
column of the paginated root entity into account (#7820, fixed in
#7821).
In order to perform this type conversion, we need to know the DBAL type
class for the root entity's `#[Id]`, and we have to figure it out based
on a given (arbitrary) DQL query. This requires DQL parsing and
inspecting the AST, so #7821 placed the conversion code in the
`WhereInWalker` where all the necessary information is available.
The problem is that type conversion has to happen every time the
paginator is run, but the query that results from running
`WhereInWalker` would be kept in the query cache. This was reported in
#7837 and fixed by #7865, by making this particular query expire every
time. The query must not be cached, since the necessary ID type
conversion happens as a side-effect of running the `WhereInWalker`.
The Paginator internal query that uses `WhereInWalker` has its DQL
re-parsed and transformed in every request.
This PR moves the code that determines the DBAL type out of
`WhereInWalker` into a dedicated SQL walker class, `RootTypeWalker`.
`RootTypeWalker` uses a ~hack~ clever trick to report the type back: It
sets the type as the resulting "SQL" string. The benefit is that
`RootTypeWalker` results can be cached in the query cache themselves.
Only the first time a given DQL query has to be paginated, we need to
run this walker to find out the root entity's ID type. After that, the
type will be returned from the query cache.
With the type information being provided, `Paginator` can take care of
the necessary conversions by itself. This happens every time the
Paginator is used.
The internal query that uses `WhereInWalker` can be cached again since
it no longer has side effects.
Since #10411 has been merged, no more need to specify all intermediate
abstract entity classes.
Thus, we can relax the schema validator check as requested in #9095. The
reasons given in #9142 no longer apply.
Fixes#9095
This spares us from referencing ClassMetadataInfo from other classes,
which is a good thing since it is deprecated. It also means merging up
is easier.
In #10431, some invalid inheritance declarations in our test base were fixed. However, the change missed to update XML, PHP and static PHP mapping configurations as well.
Apparently, this did not raise any flags because the misconfiguration only caused a deprecation notice in 2.15.x. Running the tests against 3.0 (where the misconfiguration will be an error) unveiled the mistake.
Inheritance has to be declared as soon as one entity class extends (directly or through middle classes) another one.
This is also pointed out in the opening comment for #8348:
> Entities are not allowed to extend from entities without an inheritence mapping relationship (Single Table or Joined Table inheritance). [...] While Doctrine so far allowed these things, they are fragile and will break on certain scenarios.
Throwing an exception in case of this misconfiguration is nothing we should do light-heartedly, given that it may surprise users in a bugfix or feature release. So, we should start with a deprecation notice and make this an exception in 3.0. The documentation is updated accordingly at #10429.
Catching missing inheritance declarations early on is important to avoid weird errors further down the road, giving users a clear indication of the root cause.
In case you are affected by this, please understand that although things "previously worked" for you, you have been using the ORM outside of what it was designed to do. That may have worked in simple cases, but may also have caused invalid results (wrong or missing data after hydration?) that possibly went unnoticed in subtle cases.
This picks the test case from #9517 and rebases it onto 2.14.x.
The problem has been covered in #8415, so this PR closes#9517 and fixes#9516.
Co-authored-by: Robert D'Ercole <bobdercole@gmail.com>
This PR prevents the Paginator from causing OpCache "wasted memory" to increase _on every request_ when used with Symfony's `PhpFilesAdapter` as the cache implementation for the query cache.
Depending on configured thresholds, wasted memory this will either cause periodic opcache restarts or running out of memory and not being able to cache additional scripts ([Details](https://tideways.com/profiler/blog/fine-tune-your-opcache-configuration-to-avoid-caching-suprises)).
Fixes#9917, closes#10095.
There is a long story (#7820, #7821, #7837, #7865) behind how the Paginator can take care of DBAL type conversions when creating the pagination query. This conversion has to transform identifier values before they will be used as a query parameter, so it has to happen every time the Paginator is used.
For reasons, this conversion happens inside `WhereInWalker`. Tree walkers like this are used only during the DQL parsing/AST processing steps. Having a DQL query in the query cache short-cuts this step by fetching the parsing/processing result from the cache.
So, to make sure the conversion happens also with the query cache being enabled, this line
1753d03500/lib/Doctrine/ORM/Tools/Pagination/Paginator.php (L165)
was added in #7837. It causes `\Doctrine\ORM\Query::parse()` to re-parse the query every time, but will also put the result into the query cache afterwards.
At this point, the setup described in #9917 – which, to my knowledge, is the default in Symfony + DoctrineBundle projects – will ultimately bring us to this code:
4b3391725f/src/Symfony/Component/Cache/Adapter/PhpFilesAdapter.php (L248-L249)
When writing a cache item with an already existing key, the driver has to make sure the opcache will honor the changed PHP file. This is what causes _wasted memory_ to increase.
Instead of using `\Doctrine\ORM\Query::expireQueryCache()`, which will force `\Doctrine\ORM\Query::parse()` to parse the query again before putting it into the cache, use `\Doctrine\ORM\Query::useQueryCache(false)`. The subtle difference is the latter will not place the processed query in the cache in the first place.
A test case is added to check that repeated use of the paginator does not call the cache to update existing keys. That should suffice to make sure we're not running into the issue, while at the same time not complicating tests by using the `PhpFilesAdapter` directly.
Note that in order to observe the described issue in tests, you will need to use the `PhpFilesDriver` and also make sure that OpCache is enabled and also activated for `php-cli` (which is running the unit tests).
This particular subquery generated/used by the Paginator is not put into the query cache. The DQL parsing/to-SQL conversion has to happen _every time_ the Paginator is used.
This, however, was already the case before this PR. In other words, this PR only changes that we do not store/update the cached result every time, but instead completely omit caching the query.
This should be replaced with a DTO in the next major.
To be able to have something usable, I had to move the validation of the
DTO (checking that it has a "class" attribute) to mapEmbedded, which
happens earlier than doLoadMetadata(). It is unclear to me why it was
not put here in the first place.
* Fixup GH8127 test case
This removes the unnecessary "middle2" class and puts back in the effect of the data provider.
Both changes were accidentally committed when I was working on #10411 and just meant as experiments during debugging.
* Fix CS
Looking at usages in the codebase, it is supposed to be at least
bool|string, but not string.
When dumping the value inside `getFieldMapping`, it is always a boolean.
This fixes two closely related bugs.
1. When inheriting a to-one association from a mapped superclass, update the `sourceEntity` class name to the current class only when the association is actually _declared_ in the mapped superclass.
2. Reject association types that are not allowed on mapped superclasses only when they are actually _declared_ in a mapped superclass, not when inherited from parent classes.
Currently, when a many-to-one association is inherited from a `MappedSuperclass`, mapping information will be updated so that the association has the current (inheriting) class as the source entity.
2138cc9383/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php (L384-L393)
This was added in 7dc8ef1db9 for [DDC-671](https://github.com/doctrine/orm/issues/5181).
The reason for this is that a mapped superclass is not an entity itself and has no table.
So, in the database, associations can only be from the inheriting entities' tables towards the referred-to target. This is also the reason for the limitation that only to-one associations may be added in mapped superclasses, since for those the database foreign key can be placed on the table(s) of the inheriting entities (and there may be more than one child class).
Neither the decision to update the `sourceEntity` nor the validation check should be based on `$parent->isMappedSuperclass`.
This only works in the simple case where the class hierarchy is `Mapped Superclass → Entity`.
The check is wrong when we have an inheritance hierarchy set up and the class hierarchy is `Base Entity → Mapped Superclass → Child Entity`.
Bug 1: The association should keep the root entity as the source. After all, in a JTI, the root table will contain the foreign key, and we need to base joins on that table when traversing `FROM LeafClass l JOIN l.target`.
Bug 2: Do not reject the to-many association declared in the base class. It is ok to have the reverse (owning) side point back to the base entity, as it would be if there were no mapped superclasses at all. The mapped superclass does not declare, add or otherwise interfere with the to-many association at all.
Base the decision to change the `sourceEntity` on `$mapping['inherited']` being set. This field points to the topmost _parent entity_ class in the ancestry tree where the relationship mapping appears for the first time.
When it is not set, the current class is the first _entity_ class in the hierarchy with that association. Since we are inheriting the relation, it must have been added in a mapped superclass above, but was not yet present in the nearest parent entity class.
In that case, it may only be a to-one association and the source entity needs to be updated.
(See #10396 for a clarification of the semantics of `inherited`.)
Here is a simplified example of the class hierarchy.
See the two tests added for more details – one is for checking the correct usage of a to-one association against/with the base class in JTI. The other is to test that a to-many association on the base class is not rejected.
I am sure that there are other tests that (still) cover the update of `sourceEntity` is happening.
```php
/**
* @Entity
*/
class AssociationTarget
{
/**
* @Column(type="integer")
* @Id
* @GeneratedValue
*/
public $id;
}
/**
* @Entity
* @InheritanceType("JOINED")
* @DiscriminatorColumn(name="discriminator", type="string")
* @DiscriminatorMap({"1" = "BaseClass", "2" = "LeafClass"})
*/
class BaseClass
{
/**
* @Column(type="integer")
* @Id
* @GeneratedValue
*/
public $id;
/**
* @ManyToOne(targetEntity="AssociationTarget")
*/
public $target;
}
/**
* @MappedSuperclass
*/
class MediumSuperclass extends BaseClass
{
}
/**
* @Entity
*/
class LeafClass extends MediumSuperclass
{
}
```
When querying `FROM LeafClass l`, it should be possible to `JOIN l.target`. This currently leads to an SQL error because the SQL join will be made via `LeafClass.target_id` instead of `BaseClass.target_id`. `LeafClass` is considered the `sourceEntity` for the association – which is wrong–, and so the foreign key field is expected to be in the `LeafClass` table (using JTI here).
Fixes#5998, fixes#7825.
I have removed the abstract entity class, since it is not relevant for the issue and took the discussion off course. Also, the discriminator map now contains all classes.
Added the second variant of the bug, namely that a to-many association would wrongly be rejected in the same situation.
The ArgumentTypeCoercion one is hiding issues we could address.
Addressing them will require changes that should go in the next minor,
so I'm moving them to the baseline instead. That way, we cannot
introduce more issues of this type in this file.
* 2.14.x:
PHPStan 1.9.8, Psalm 5.4.0 (#10382)
fix typo for missing a comma (#10377)
Docs: Removing `type: 'integer'` from mappings (#10368)
Docs: Moving *attributes* mapping to first position (#10364)
Docs: Deleting duplicate mapping example (#10363)
Yet another micro-PR ;-) - as requested at https://github.com/doctrine/orm/pull/10364#issuecomment-1370155521
I also changed `$currentPrice` from `float` to `int`, since IMO it's better to store prices as `int` (=cents).
Question: Is there a reason why most typehints are `int|null`, instead of `?int`? Should I change them?
Following 1915dcd1e8, 0 is now used as a
default value, and Query::$firstResult is no longer nullable, but it
seems getFirstResult() was overlooked. The class is final, so this is no
breaking change.
* 2.14.x:
Shorter deprecation message (#10357)
Add Fully-Qualified class name in UnrecognizedField exception to ease debugging (#10342)
Include parameter types in hydration cache key generation (#10355)
It is important to have the same version of all dependencies in dev and
in the CI, otherwise it makes it hard to have the right static analysis
baseline for every environment.
Since doctrine/annotations 1.10, autoloading is used when everything
else has failed. Using registerFile() has been deprecated in favor of
that later on.
Previously, only a predefined set of automatic mappings was allowed, such as int, float, boolean, DateTime etc.
With this extension, it is possible to supply custom TypedFieldMapper implementation which takes as parameter the ReflectionProperty of a given field and decides the appropriate mapping.
A new configuration option was added to set and get the TypedFieldMapper.
The old logic was moved into a class DefaultTypedFieldMapper which is used by default when no mapper is supplied.
The selected TypedFieldMapper is passed into ClassMetadataInfo constructor. If empty, the DefaultTypedFieldMapper is used.
There is also ChainTypedFieldMapper class which allows chaining multiple TypedFieldMappers and apply them in a cascade (always if a field gets type assigned by the earlier mapper in the list, it will not be changed later).
This commit adds enumType option to DiscriminatorColumn as well as support for custom DBAL types which use PHP enums for PHP values.
Previously, the enumType option was completely missing, but also even using custom types that used PHP enums would end up in exception because ObjectHydrator would try to convert enums to string using (string) explicit conversion.
Apart from hydrators, ClassMetadataBuilder was extended to support specifying enumType.
Documentation was updated.
Previously, array of enums were incorrectly compared in UoW::computeChangeSet() resulting always in false positive, since the original data for comparison is fetched using ReflectionEnumProperty::getValue(), which returns the enum values, not enum objects.
This fix ensures comparing the individual enum array members' values.
Enum fields as ID have worked for some time, but referencing these fields from other entities in association mappings such as OneToOne, ManyToOne and ManyToMany resulted in fatal error as there was an attempt to convert enum value to string improperly.
Previously different hydrators would store the original data for enum
fields in different ways, the SimpleObjectHydrator would keep them as
strings while other hydrators would convert then to native php enums.
This would make the data in the internal UnitOfWork::$originalEntityData
array inconsistent which could've caused problems in the long run.
Now, all hydrators convert enum fields to native php enums ensuring the
original data is always consistent regardless of the hydrator used.
In 68bc00b6c6, while fixing coding style,
I introduced many, many use statements in that file. Using
half-qualified names sometimes, and unqualified names some other times
makes no sense.
If AST\ is used at least once, use it always.
It is not stable yet, but should be good enough, and this will help
taking care of #10118
Let us baseline all the new issues, just because they are new does not
mean they are more important than already-baselined errors. Besides, it
is more important to have higher standards for new code than to get an
increased baseline.
This exception is used in two places, one where $generatedMode is
clearly a string, and another where typing it as a string will cause a
type error, because $generatedMode is supposed to be an int there.
Inside the Query\AST namespace, many classes use public properties that
are supposed to be set from outside the class. Let us ignore
PropertyNotSetInConstructor for that entire namespace instead of doing
it on a case by case basis.
stderr is not a great name for something that is not meant to be
processed (piped into) a program, but to be read by humans.
Most commands should use stderr, and some of them should partially use
stdout, typically when dumping SQL requests.
The previous rework missed some details / left the documentation in an
inconsistent state. Searching for "annotation" case insensitively allows
to find discrepancies and forgotten things.
- migrate to attributes;
- add helpful phpdoc annotations;
- use typed properties;
- add type declarations.
Co-authored-by: Alexander M. Turek <me@derrabus.de>
The downside of this is that we will have to tweak the settings so that
no job is required anymore.
The upside is that builds should be faster, and less resource-intensive.
__sleep() does not list these properties as suposed to be serialized.
This means we have to assert that it is not in the many scenarios where
we resort to these properties. Introducing a private method allows us to
centralize the assertions.
* Add missing variable name in param phpdoc
* Use psalm-assert instead of psalm-param
The parameter can in fact be any string, but if an exception is not
thrown, then it was a valid value.
Deprecating this class means that users using SA tools have to
update their code specifying the ObjectManager implementation.
Instead of this, dedicated classes for each event should be
created before deprecating this class.
the description and semantics of the `--dump-sql` switch
indicate and sql dump. without that useless line
The following SQL statements will be executed:
the output can actually be used to generate a plain
dump.sql file that can be fed into sql command-
consuming programs.
resolves#8263 and #7186.
doctrine/dbal is the component responsible for generating the queries.
Let us make the test suite more robust by asserting that things work
from a functional point of view.
* 2.12.x:
PHPStan 1.8.0 (#9887)
Fix typo in AbstractQuery
ObjectHydrator: defer initialization of potentially empty collections
Migrate more usages of SchemaTool::createSchema()
preUpdate: Add restriction that changed field needs to be in computed changeset (#9871)
*FlushEventArgs classes should extend ManagerEventArgs from
doctrine/persistence to be able to use ManagerEventArgs for any
persistance implementation.
OnClearEventArgs should extend from OnClearEventArgs from
doctrine/persistence.
These tests were using the fact that some arguments of some methods of
the naming strategy interface are optional or nullable for now to avoid
providing some. In practice, these arguments are always provided, and
that should also be the case in tests.
When computing a foreign key column name, the referenced column name
may be null in the case of a self referencing entity with join columns
defined in the mapping.
@@ -375,7 +855,7 @@ If your code relies on single entity flushing optimisations via
Said API was affected by multiple data integrity bugs due to the fact
that change tracking was being restricted upon a subset of the managed
entities. The ORM cannot support committing subsets of the managed
entities. The ORM cannot support committing subsets of the managed
entities while also guaranteeing data integrity, therefore this
utility was removed.
@@ -476,8 +956,8 @@ either:
- map those classes as `MappedSuperclass`
## Minor BC BREAK: ``EntityManagerInterface`` instead of ``EntityManager`` in type-hints
As of 2.5, classes requiring the ``EntityManager`` in any method signature will now require
As of 2.5, classes requiring the ``EntityManager`` in any method signature will now require
an ``EntityManagerInterface`` instead.
If you are extending any of the following classes, then you need to check following
signatures:
@@ -570,7 +1050,7 @@ the `Doctrine\ORM\Repository\DefaultRepositoryFactory`.
When executing DQL queries with new object expressions, instead of returning DTOs numerically indexes, it will now respect user provided aliases. Consider the following query:
SELECT new UserDTO(u.id,u.name) as user,new AddressDTO(a.street,a.postalCode) as address, a.id as addressId FROM User u INNER JOIN u.addresses a WITH a.isPrimary = true
Previously, your result would be similar to this:
array(
@@ -808,7 +1288,7 @@ The EntityRepository now has an interface Doctrine\Persistence\ObjectRepository.
The annotation reader was heavily refactored between 2.0 and 2.1-RC1. In theory the operation of the new reader should be backwards compatible, but it has to be setup differently to work that way:
As you can see, we have a method "setBlockEntity" which ties a potential strategy to an object of type AbstractBlock. This type will simply define the basic behaviour of our blocks and could potentially look something like this:
..code-block::php
<?php
@@ -154,8 +154,8 @@ As you can see, we have a method "setBlockEntity" which ties a potential strateg
* This var contains the classname of the strategy
* that is used for this blockitem. (This string (!) value will be persisted by Doctrine ORM)
*
* This is a doctrine field, so make sure that you use an @column annotation or setup your
* yaml or xml files correctly
* This is a doctrine field, so make sure that you use a
#[Column] attribute or setup your yaml or xml files correctly
* @var string
*/
protected$strategyClassName;
@@ -177,7 +177,7 @@ As you can see, we have a method "setBlockEntity" which ties a potential strateg
publicfunctiongetStrategyClassName(){
return$this->strategyClassName;
}
/**
* Returns the instantiated strategy
*
@@ -186,7 +186,7 @@ As you can see, we have a method "setBlockEntity" which ties a potential strateg
publicfunctiongetStrategyInstance(){
return$this->strategyInstance;
}
/**
* Sets the strategy this block / panel should work as. Make sure that you've used
* this method before persisting the block!
@@ -213,28 +213,29 @@ This might look like this:
..code-block::php
<?php
use\Doctrine\ORM,
\Doctrine\Common;
useDoctrine\Common\EventSubscriber;
useDoctrine\ORM\Event\LifecycleEventArgs;
useDoctrine\ORM\Events;
/**
* The BlockStrategyEventListener will initialize a strategy after the
but be aware that it is deprecated since PHP 8.1. We do not recommend its usage.
- PHP 7.4 introduces :doc:`the new magic method <https://php.net/manual/en/language.oop5.magic.php#object.unserialize>`
``__unserialize``, which changes the execution priority between
``__wakeup`` and itself when used. This can cause unexpected behaviour in
an Entity.
- An entity class must not be final nor read-only but
it may contain final methods or read-only properties.
- Any two entity classes in a class hierarchy that inherit
directly or indirectly from one another must not have a mapped
property with the same name. That is, if B inherits from A then B
must not have a mapped field with the same name as an already
mapped field that is inherited from A.
- An entity cannot make use of func_get_args() to implement variable parameters.
Generated proxies do not support this for performance reasons and your code might
actually fail to work when violating this restriction.
- Entity cannot access private/protected properties/methods of another entity of the same class or :doc:`do so safely <../cookbook/accessing-private-properties-of-the-same-class-from-different-instance>`.
Entities support inheritance, polymorphic associations, and
polymorphic queries. Both abstract and concrete classes can be
@@ -113,6 +100,25 @@ classes, and non-entity classes may extend entity classes.
never calls entity constructors, thus you are free to use them as
you wish and even have it require arguments of any type.
Mapped Superclasses
~~~~~~~~~~~~~~~~~~~
A mapped superclass is an abstract or concrete class that provides
persistent entity state and mapping information for its subclasses,
but which is not itself an entity.
Mapped superclasses are explained in greater detail in the chapter
on :doc:`inheritance mapping </reference/inheritance-mapping>`.
Transient Classes
~~~~~~~~~~~~~~~~~
The term "transient class" appears in some places in the mapping
drivers as well as the code dealing with metadata handling.
A transient class is a class that is neither an entity nor a mapped
superclass. From the ORM's point of view, these classes can be
completely ignored, and no class metadata is loaded for them at all.
Entity states
~~~~~~~~~~~~~
@@ -159,17 +165,13 @@ Serializing entities
Serializing entities can be problematic and is not really
recommended, at least not as long as an entity instance still holds
references to proxy objects or is still managed by an
EntityManager. If you intend to serialize (and unserialize) entity
instances that still hold references to proxy objects you may run
into problems with private properties because of technical
limitations. Proxy objects implement ``__sleep`` and it is not
possible for ``__sleep`` to return names of private properties in
parent classes. On the other hand it is not a solution for proxy
objects to implement ``Serializable`` because Serializable does not
work well with any potential cyclic object references (at least we
did not find a way yet, if you did, please contact us). The
``Serializable`` interface is also deprecated beginning with PHP 8.1.
references to proxy objects or is still managed by an EntityManager.
By default, serializing proxy objects does not initialize them. On
unserialization, resulting objects are detached from the entity
manager and cannot be initialized anymore. You can implement the
``__serialize()`` method if you want to change that behavior, but
then you need to ensure that you won't generate large serialized
object graphs and take care of circular associations.
The EntityManager
~~~~~~~~~~~~~~~~~
@@ -203,5 +205,3 @@ typical implementation of the
to keep track of all the things that need to be done the next time
``flush`` is invoked. You usually do not directly interact with a
``UnitOfWork`` but with the ``EntityManager`` instead.
@@ -180,10 +180,10 @@ not need to lazy load the association with another query.
Doctrine allows you to walk all the associations between
all the objects in your domain model. Objects that were not already
loaded from the database are replaced with lazyload proxy
instances. Non-loaded Collections are also replaced by lazy-load
loaded from the database are replaced with lazy-loading proxy
instances. Non-loaded Collections are also replaced by lazy-loading
instances that fetch all the contained objects upon first access.
However relying on the lazy-load mechanism leads to many small
However relying on the lazy-loading mechanism leads to many small
queries executed against the database, which can significantly
affect the performance of your application. **Fetch Joins** are the
solution to hydrate most or all of the entities that you need in a
@@ -319,11 +319,11 @@ With Nested Conditions in WHERE Clause:
<?php
$query=$em->createQuery('SELECT u FROM ForumUser u WHERE (u.username = :name OR u.username = :name2) AND u.id = :id');
$query->setParameters(array(
$query->setParameters([
'name'=>'Bob',
'name2'=>'Alice',
'id'=>321,
));
]);
$users=$query->getResult();// array of ForumUser objects
With COUNT DISTINCT:
@@ -464,6 +464,11 @@ hierarchies:
$query=$em->createQuery('SELECT u FROM Doctrine\Tests\Models\Company\CompanyPerson u WHERE u INSTANCE OF Doctrine\Tests\Models\Company\CompanyEmployee');
$query=$em->createQuery('SELECT u FROM Doctrine\Tests\Models\Company\CompanyPerson u WHERE u INSTANCE OF ?1');
$query=$em->createQuery('SELECT u FROM Doctrine\Tests\Models\Company\CompanyPerson u WHERE u NOT INSTANCE OF ?1');
@@ -28,10 +28,11 @@ table alias of the SQL table of the entity.
In the case of joined or single table inheritance, you always get passed the ClassMetadata of the
inheritance root. This is necessary to avoid edge cases that would break the SQL when applying the filters.
Parameters for the query should be set on the filter object by
``SQLFilter#setParameter()``. Only parameters set via this function can be used
in filters. The ``SQLFilter#getParameter()`` function takes care of the
proper quoting of parameters.
For the filter to correctly function, the following rules must be followed. Failure to do so will lead to unexpected results from the query cache.
1. Parameters for the query should be set on the filter object by ``SQLFilter#setParameter()`` before the filter is used by the ORM ( i.e. do not set parameters inside ``SQLFilter#addFilterConstraint()`` function ).
2. The filter must be deterministic. Don't change the values based on external inputs.
The ``SQLFilter#getParameter()`` function takes care of the proper quoting of parameters.
@@ -612,10 +622,10 @@ Now two examples of what happens when you remove the references:
$em->flush();
In this case you have not only changed the ``Contact`` entity itself but
you have also removed the references for standing data and as well as one
address reference. When flush is called not only are the references removed
but both the old standing data and the one address entity are also deleted
In this case you have not only changed the ``Contact`` entity itself but
you have also removed the references for standing data and as well as one
address reference. When flush is called not only are the references removed
but both the old standing data and the one address entity are also deleted
from the database.
.._filtering-collections:
@@ -708,6 +718,7 @@ methods:
*``andX($arg1, $arg2, ...)``
*``orX($arg1, $arg2, ...)``
*``not($expression)``
*``eq($field, $value)``
*``gt($field, $value)``
*``lt($field, $value)``
@@ -725,6 +736,35 @@ methods:
..note::
There is a limitation on the compatibility of Criteria comparisons.
You have to use scalar values only as the value in a comparison or
the behaviour between different backends is not the same.
Depending on whether the collection has already been loaded from the
database or not, criteria matching may happen at the database/SQL level
or on objects in memory. This may lead to different results and come
surprising, for example when a code change in one place leads to a collection
becoming initialized and, as a side effect, returning a different result
or even breaking a ``matching()`` call somewhere else. Also, collection
initialization state in practical use cases may differ from the one covered
in unit tests.
Database level comparisons are based on scalar representations of the values
stored in entity properties. The field names passed to expressions correspond
to property names. Comparison and sorting may be affected by
database-specific behavior. For example, MySQL enum types sort by index position,
not lexicographically by value.
In-memory handling is based on the ``Selectable`` API of `Doctrine Collections <https://www.doctrine-project.org/projects/doctrine-collections/en/stable/index.html#matching>`.
In this case, field names passed to expressions are being used to derive accessor
method names. Strict type comparisons are used for equal and not-equal checks,
and generally PHP language rules are being used for other comparison operators
or sorting.
As a general guidance, for consistent results use the Criteria API with scalar
values only. Note that ``DateTime`` and ``DateTimeImmutable`` are two predominant
examples of value objects that are *not* scalars.
Refrain from using special database-level column types or custom Doctrine Types
that may lead to database-specific comparison or sorting rules being applied, or
to database-level values being different from object field values.
Provide accessor methods for all entity fields used in criteria expressions,
and implement those methods in a way that their return value is the
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.