This command's purpose is to provide structured data, except for a call
to caution() that warns the user in case they do not have any mapped
entities or they have errors.
Symfony 7.3 is not available to all of our users, so we cannot switch to
native lazy objects, which require a PHP version higher than the lowest
PHP version we support.
In f256d996cc, I did a global move to
stderr for notifications, and went a bit overboard for
MappingDescribeCommand, which purpose is to output a description.
For some reason, it does not appear to work when nested inside a
code-block directive. Anyway, if you specify the language attribute, you
get markup identical to what you obtain when using code-block and
literalinclude, so this wrapping seems unneeded.
PostLoad listeners might initialize values for transient properties, so
the proxy should not skip initialization when using transient
properties.
Co-authored-by: Nicolas Grekas <nicolas.grekas@gmail.com>
The old proxy implementation of doctrine/common was triggered by public
methods rather than access to properties (making public properties
unsupported in entities), so tests could use public instance properties
to track the state of postLoad lifecycle callbacks without triggering
the proxy initialization when reading that state (which then changes the
state of triggering the postLoad callback).
As the new proxy implementation hooks into properties instead, the tests
now use a static method (ensuring it is reset properly before loading
the instance for which we care about the tracking) instead of an
instance property.
* Introduce PHP 8.4 lazy proxy/ghost API.
* Call setRawValueWithoutLazyInitialization for support with lazy proxy.
* Refactorings
* Revert test change partially and skip with lazy objects.
* Houskeeping: phpcs
* Run with ENABLE_LAZY_PROXY=1 in php 8.4 matrix.
* Fix ci
* Transient properties are not skipping lazy initialization anymore, to expensive and could lead to errors. Adjust lifecycle test that uses transient properittes for assertions.
* Restore behavior preventing property hook use in 8.4 in unsupported coditions
* Add \ReflectionClass::SKIP_INITIALIZATION_ON_SERIALIZE
Co-authored-by: Nicolas Grekas <nicolas.grekas@gmail.com>
* Rename isNativeLazyObjectsEnabled/enableNativeLazyObjects.
* Housekeeping: phpcs
* Update advanced-configuration docs and make proxy config variables not required anymore with native lazy objects.
* Move code around
* Apply suggestions from code review
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
* Pick suggestions
---------
Co-authored-by: Nicolas Grekas <nicolas.grekas@gmail.com>
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
The dev configuration and CI configuration should not diverge this much.
I do not think the current situation was intended. A difference that
remains after my changes is the bootstrap file, which in dev seems aimed
at helping contributors setup their environment.
This fixes a bug that arises when using Pagination and an entity relation is mapped with fetch-mode EAGER but setFetchMode LAZY (or anything that is not EAGER) has been used on the query. If the query use WITH condition, an exception is incorrectly raised (Associations with fetch-mode=EAGER may not be using WITH conditions).
The class LimitSubqueryOutputWalker clones the query, but not its parameters and hints, so the generated subquery does not know that fetch-mode has been overridden.
Fixes#11741
The bug related (#11694) and fixed mapping of sql column alias to field in entity (#11783) and
invalidate cache [cache/persisted/entity|cache/persisted/collection] when sql filter changes
Previously, when using a custom naming strategy, explicitly declaring a JoinColumn required specifying the referencedColumnName always as it would default to id no matter the naming strategy. This PR changes it to be determines correctly.
Ref #9558
When there are no conflicts between branches, we create pull requests
where the head branch is a branch on the origin repository. That branch
points to a commit that should already have coverage information
provided by the build that happens after merging a regular pull request.
The thing is, coverage information provided by builds that happen before
merging a pull request are associated with the commit of the head
repository. This means that when merging up 1.2 into 1.3, the build
produces coverage information that is the result of a merge between 1.2
and 1.3, and associates it with 1.2, although it is run on with a
codebase that is much closer to 1.3 (and is in fact supposed to become
1.3 after the merge).
This means that when we create a merge up PR from 1.2 to anything else,
the coverage information is going to be wrong until a PR targeting 1.2
gets merged.
I do not think we need coverage about conflictless merge up PRs more
than we need accurate numbers, so I propose we disable the upload for
those instead of, say, trying to associate them with the temporary merge
commit.
CachedPersisterContext::$selectJoinSql should be clear or regenerated when sqlFilter changed
The problem reproduce when in use fetch=EAGER and use additional sql filter on this property
I think it would be great to use literalinclude for big code snippets,
because our IDEs could warn us about issues, and it would be easily to
showcase our coding standard. Before we do that though, let us validate
that it renders as expected. I have picked a complex example where we
have a configuration block.
* Add a test covering the #11112 issue
* Add new OutputWalker and SqlFinalizer interfaces
* Add a SingleSelectSqlFinalizer that can take care of adding offset/limit as well as locking mode statements to a given SQL query.
Add a FinalizedSelectExecutor that executes given, finalized SQL statements.
* In SqlWalker, split SQL query generation into the two parts that shall happen before and after the finalization phase.
Move the part that generates "pre-finalization" SQL into a dedicated method. Use a side channel in SingleSelectSqlFinalizer to access the "finalization" logic and avoid duplication.
* Fix CS violations
* Skip the GH11112 test while applying refactorings
* Avoid a Psalm complaint due to invalid (?) docblock syntax
* Establish alternate code path - queries can obtain the sql executor through the finalizer, parser knows about output walkers yielding finalizers
* Remove a possibly premature comment
* Re-enable the #11112 test
* Fix CS
* Make RootTypeWalker inherit from SqlOutputWalker so it becomes finalizer-aware
* Update QueryCacheTest, since first/max results no longer need extra cache entries
* Fix ParserResultSerializationTest by forcing the parser to produce a ParserResult of the old kind (with the executor already constructed)
* Fix WhereInWalkerTest
* Update lib/Doctrine/ORM/Query/Exec/PreparedExecutorFinalizer.php
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
* Fix tests
* Fix a Psalm complaint
* Fix a test
* Fix CS
* Make the NullSqlWalker an instance of SqlOutputWalker
* Avoid multiple cache entries caused by LimitSubqueryOutputWalker
* Fix Psalm complaints
* Fix static analysis complaints
* Remove experimental code that I committed accidentally
* Remove unnecessary baseline entry
* Make AddUnknownQueryComponentWalker subclass SqlOutputWalker
That way, we have no remaining classes in the codebase subclassing SqlWalker but not SqlOutputWalker
* Use more expressive exception classes
* Add a deprecation message
* Move SqlExecutor creation to ParserResult, to minimize public methods available on it
* Avoid keeping the SqlExecutor in the Query, since it must be generated just in time (e. g. in case Query parameters change)
* Address PHPStan complaints
* Fix tests
* Small refactorings
* Add an upgrade notice
* Small refactorings
* Update the Psalm baseline
* Add a missing namespace import
* Update Psalm baseline
* Fix CS
* Fix Psalm baseline
---------
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
It maybe happen that the SQL COMMIT statement is successful, but then
something goes wrong. In that kind of case, you do not want to attempt a
rollback.
This was implemented in UnitOfWork::commit(), but for some reason not in
the similar EntityManager methods.
We use this method only from within one of our own test cases, and I don't see how it would be useful to anybody else outside – it has to be called on the `Parser` instance which exists internally in the `Query` only.
Deprecating and removing it in 3.x allows for a slight simplification in the `Parser` there, since we do no longer need the field (it can be a local variable).
People that might have experimented with property hooks while still
using ORM < 2.20.0 need to know that they need to remove their
experiment or upgrade to a version that explicitly supports them.
* 2.19.x:
Make nullable parameters explicit in generated entities (#11625)
Update attributes-reference.rst
Bump doctrine/.github from 5.0.1 to 5.1.0 (#11616)
Move orphan metadata to where it belongs
PHPStan 1.12 (#11585)
When adding the same lifecycle event callback to two or more lifecycle events, the generator will create a stub for each event resulting in fatal 'Cannot redeclare' errors. That is, only if the callback name contains uppercase letters.
If the source entity for an inverse (non-owning) 1-1 relationship is
identified by an association then the identifying association may not
be set when an inverse one-to-one association is resolved. This means
that no data is available in the entity to resolve the needed column
value for the join query.
The original entity data can be retrieved from the unit of work and
is used as a fallback to populate the query condition.
Fixes#11108
single-inheritence entity parent as targetEntity.
When using the parent entity for a single-inheritence table as the
targetEntity for a property, the discriminator value should be all
of the values in the discriminator map.
OneToManyPersister::deleteEntityCollection has been amended to
reflect this.
This change makes CountWalker use COUNT(*) instead of
COUNT(tbl.id), when the user declared that their query
does not need to use (SELECT) DISTINCT, which is
commonly the case when there are no JOINs in the query,
or when the JOINs are only *ToOne.
Research showed that COUNT(*) allows databases to use
index(-only) scans more eagerly from any of the
indexed columns, especially when the query is using
a WHERE-condition that filters on an indexed column.
The Query class (used for DQL queries) takes care of using the value and
type as is when a type was specified for a parameter instead of going
through the default processing of values.
The NativeQuery class was missing the equivalent check, making the
custom type work only if the default processing of values does not
convert the value to a different one.
When a ManyToOne attribute is encountered on an Embeddable class, the exception message reads "Attribute "Doctrine\ORM\Mapping\OneToMany" on embeddable [class] is not allowed.". This should be "Doctrine\ORM\Mapping\ManyToOne" on embeddable [class] is not allowed.".
When using ghost objects, the method was leaking a `static` return type due to the way it was implemented, which is incompatible with the native return type that will be added in doctrine/persistence v4.
Remove redundant condition to check if target class contains foreign
identifier in order to allow fetching a null for relations with
composite keys, when part of the key value is null.
* 2.19.x:
Fix OneToManyPersister::deleteEntityCollection missing discriminator column/value. (GH-11500)
Skip joined entity creation for empty relation (#10889)
ci: maintained and stable mariadb version (11.4 current lts) (#11490)
fix(docs): use string value in `addAttribute`
Replace assertion with exception (#11489)
Use ramsey/composer-install in PHPBench workflow
update EntityManager#transactional to EntityManager#wrapInTransaction
Fix cloning entities
Consider usage of setFetchMode when checking for simultaneous usage of fetch-mode EAGER and WITH condition.
* 2.19.x:
Fix OneToManyPersister::deleteEntityCollection missing discriminator column/value. (GH-11500)
Skip joined entity creation for empty relation (#10889)
ci: maintained and stable mariadb version (11.4 current lts) (#11490)
fix(docs): use string value in `addAttribute`
Replace assertion with exception (#11489)
Use ramsey/composer-install in PHPBench workflow
update EntityManager#transactional to EntityManager#wrapInTransaction
Fix cloning entities
Consider usage of setFetchMode when checking for simultaneous usage of fetch-mode EAGER and WITH condition.
This fixes a bug that arises when an entity relation is mapped with
fetch-mode EAGER but setFetchMode LAZY (or anything that is not EAGER)
has been used on the query. If the query use WITH condition, an
exception is incorrectly raised (Associations with fetch-mode=EAGER may
not be using WITH conditions).
Fixes#11128
Co-Authored-By: Albert Prat <albert.prat@interactiu.cat>
* 3.1.x:
Fix failed merge (#11464)
Test with actual lock modes (#11465)
Backport test for Query::setLockMode() (#11463)
Fix return type of Query::getLockMode() (#11462)
* 3.1.x:
Using an integer as discriminator value with ORM v3
Using an integer as discriminator value with ORM v3
Bump ramsey/composer-install from 2 to 3 (#11442)
Use ramsey/composer-install in PHPBench workflow (#11444)
Bump doctrine/.github from 3.0.0 to 5.0.1
Upgrade codecov/codecov-action
Setup Dependabot
This fixes a bug that occurred when configuring integers as discriminator values and using DQL instanceOf function in the queries. Doctrine throws a type error whenever the application generates these queries.
This fixes a bug that occurred when configuring integers as discriminator values. Doctrine throws a type error whenever the application generates queries.
Targeting 2.19.x, since we want the updates to bubble up. Since
Dependabot has had no effect on doctrine/dbal yet, I suppose that means
that "dependabot.yml" must be present on the default branch.
If the entity gets reloaded from database before the deletions are
executed UnitOfWork needs to be able to return the original instance in
REMOVED state.
Changed capitalized column names to lowercase for consistency. Other occurances of column names mentioned as lowercase several times at this same page.
See https://github.com/doctrine/collections/issues/368 for the same
issue in doctrine/collections which has been fixed there.
The issue happens when using ->contains(). Running psalm emits
> InvalidArgument - Argument 1 of Doctrine\ORM\PersistentCollection::contains
> expects
> TMaybeContained:fn-doctrine\common\collections\readablecollection::contains
> as mixed, but … provided.
Solution: we should either not define @template TMaybeContained or
re-define the complete psalm docblock from ReadableCollection.
Repairing the docblock necessitates an update to the psalm baseline:
one "known issue" is no longer an issue and thus removed.
* 3.1.x:
Adjust PHPBench mocks
Set column length explicitly (#11393)
Add missing import
Remove unused variable (#11391)
Fixed proxy initialization for EnumReflectionProperty
Remove older versions from the docs (#11383)
[Documentation] Removing "Doctrine Mapping Types" ... (#11384)
[GH-11185] Bugfix: do not use collection batch loading for indexBy assocations. (#11380)
Improve lazy ghost performance by avoiding self-referencing closure. (#11376)
Remove outdated git metadata files (#11362)
Switch join columns around, otherwise index doesnt match
Key on fk
Fix entities and mapping.
Minor code style fix in AbstractRemoteControl
Do not schedule batch loading for target classes with composite identifier.
Cleanup tests not to use model sets.
provides a test case for github issue 11154
The properties `indexes` and `uniqueConstraints` were used by the
`AnnotationDriver` but were never implemented for the `AttributeDriver`.
Since the `AnnotationDriver` doesn't exist anymore these can become
deprecated and will then be removed afterwards.
Change `PESSIMISTIC_READ` to `PESSIMISTIC_WRITE`. Otherwise, the solution to the race condition at the bottom of the article would allow concurrent reads, which would not solve the presented race condition problem.
* Fix loading SchemaTool::getSchemaFromMetadata() uniqueConstraint without a name
Fixes a type miss-match exception when reading a UniqueConstraint defined on an Entity which doesn't have a predefined name.
* Fix deprecation on DBAL 3
---------
Co-authored-by: Alexander M. Turek <me@derrabus.de>
* Use class from persistence package
It is meant to remove duplication between the ORM and the ODM.
* Update UPGRADE.md
Co-authored-by: Steve Todd <stodd@mashbo.com>
---------
Co-authored-by: Alexander M. Turek <me@derrabus.de>
Co-authored-by: Steve Todd <stodd@mashbo.com>
`getAssociationMappedByTargetField()` returns `null` when called with
the owning side of an association.
This is undocumented and wrong because the phpdoc advertises a string as
a return type.
Instead, callers should ensure they are calling that method with an
inverse side.
Closes#11250
In 2.x, getAssociationMappedByTargetField() used to return null when
called with the owning side of an association.
That was undocumented and wrong because the phpdoc advertises a string
as a return type.
In 6ce0cf4a3d, I wrongly assumed that
nobody would be calling this method with the owning side of an
association.
Let us throw a full fledged exception and advertise the proper way of
avoiding this situation.
Closes#11250
The phpdoc comment for the return type of
ClassMetadata::fullyQualifiedClassName() says that the return type will
be null if the input value is null. I have made it more precise by
using "if and only if", made the null check more strict and translated
that into template annotations. Also, since we say we return a
class-string, I've asserted that.
Although this method is guaranteed to return either null or something
that can be used as a fully qualified class name, it never actually
checks that the class actually exists. Adding such a check breaks
several tests, including some that expect a exceptions at some later
points in the execution.
* 3.0.x:
Test different ways of settings query parameters
Be less restrictive in DiscriminatorColumnMapping phpdoc (#11226)
Allow (Array)ParameterType in QueryBuilder
After 2.17 (some?) EAGER fetched OneToMany associations stopped working, if they have multiple join columns. Loads for these associations will trigger a `MessingPositionalParameter` exception "Positional parameter at index 1 does not have a bound value".
This test case should reproduce this issue, so it can be fixed.
The comment above mentions that on some platforms, it might return
false, and this is why there is a check in the first place. Let us do
exactly what is mentioned in the comment.
* 3.0.x:
Remove broken assertion from DateAddFunction and DateSubFunction (#11243)
Remove unused trait
[Documentation] Adding link to Postgres upgrade article (#11257)
fix: support array-type arg in QB variadic calls (#11242)
* 2.19.x:
Fix Static Analysis folder reference (#11281)
docs: recommend safer way to disable logging (#11269)
Remove unused baseline entries
Treat '0' as a legitimate trim char
Add type field mapper documentation to the sidebar
Mark document as orphan
Use correction sectionauthor syntax
Make docs valid according to guides 0.3.3 (#11252)
* Remove trailing newlines
* Recommend safer way to disable logging
Resetting the middlewares on the configuration object will only work if
the connection object hasn't been built from that configuration object
yet. Instead, people should find the logger bound to the logging
middleware and disable it.
* 3.0.x:
Revert "Merge pull request #11229 from greg0ire/add-columns"
Add columns for 3.1.x and 4.0x
Update version ORM from 2 to 3 in docs (#11221)
Clean up outdated sentence (#11224)
Update README.md
Point link to correct upgrade guide (#11220)
Ignore subclasses without discriminatorValue when generating discriminator column condition SQL (#11200)
Update branches in README
* 2.18.x:
Point link to correct upgrade guide (#11220)
Ignore subclasses without discriminatorValue when generating discriminator column condition SQL (#11200)
Update branches in README
* Add TokenType class
Co-authored-by: Alexander M. Turek <me@derrabus.de>
* Deprecated Lexer constants in favour of TokenType
* Replace all Lexer::T_ occurrences with TokenType::T_
* Add upgrade note
* Fixed import Lexer => TokenType
* Fixed deprecation phpdoc
* Replaced int value with matching constant of TokenType
* Update src/Query/Lexer.php
---------
Co-authored-by: Alexander M. Turek <me@derrabus.de>
After commit 4e8e3ef30b when `\Doctrine\ORM\Query\SqlWalker` generates dicsriminator column condition SQL (method `\Doctrine\ORM\Query\SqlWalker::generateDiscriminatorColumnConditionSQL`) it adds an empty string to the list of possible values if the inheritance hierarchy contains a non-root abstract class.
When the discriminator column is implemented with a custom type in PostgreSQL (equivalent of Enum) the query fails because the type cannot have a value of an empty string. It boils down to the fact that `\Doctrine\ORM\Mapping\ClassMetadataInfo::$subClasses` contains an abstract class and in its Metadata the value of `\Doctrine\ORM\Mapping\ClassMetadataInfo::$discriminatorValue` is `null`.
#### Previous behavior
In version 2.14.1 `\Doctrine\ORM\Mapping\ClassMetadataInfo::$subClasses` does not contain an abstract class.
Fixes#11199, fixes#11177, fixes#10846.
---------
Co-authored-by: Michael Skvortsov <michael.skvortsov@eleving.com>
Co-authored-by: Matthias Pigulla <mp@webfactory.de>
The "any" tags inside the definition for mapped superclasses and
embeddables duplicate what is already done for entities.
The other removed "any" tags are also redundant, as they duplicate
what's already done inside the grandparent "choice" tag.
Starting with version libxml 2.12, such redundant tags cause errors
about the content model not being "determinist".
Fixes#11117
When using `AttributeOverride` to override mapping information inherited from a parent class (a mapped superclass), make sure to keep information about where the field was originally declared.
This is important for `private` fields: Without the correct `declared` information, it will lead to errors when cached mapping information is loaded, reflection wakes up and looks for the private field in the wrong class.
#10927 reported that #10455 broke the way how the default `@SequenceGeneratorDefinition` is created and inherited by subclasses for ID columns using `@GeneratedValue(strategy="SEQUENCE")`.
First, I had to understand how `@SequenceGeneratorDefinition` has been handled before #10455 when entity inheritance comes into play:
* Entity and mapped superclasses inherit the ID generator type (as given by `@GeneratedValue`) from their parent classes
* `@SequenceGeneratorDefinition`, however, is not generally inherited
* ... instead, a default sequence generator definition is created for every class when no explicit configuration is given. In this case, sequence names are based on the current class' table name.
* Once a root entity has been identified, all subclasses inherit its sequence generator definition unchanged.
#### Why did #10455 break this?
When I implemented #10455, I was mislead by two tests `BasicInheritanceMappingTest::testGeneratedValueFromMappedSuperclass` and `BasicInheritanceMappingTest::testMultipleMappedSuperclasses`.
These tests check the sequence generator definition that is inherited by an entity class from a mapped superclass, either directly or through an additional (intermediate) mapped superclass.
The tests expect the sequence generator definition on the entity _to be the same_ as on the base mapped superclass.
The reason why the tests worked before was the quirky behaviour of the annotation and attribute drivers that #10455 was aiming at: The drivers did not report the `@SequenceGeneratorDefinition` on the base mapped superclass where it was actually defined. Instead, they reported this `@SequenceGeneratorDefinition` for the entity class only.
This means the inheritance rules stated above did not take effect, since the ID field with the sequence generator was virtually pushed down to the entity class.
In #10455, I did not realize that these failing tests had to do with the quirky and changed mapping driver behaviour. Instead, I tried to "fix" the inheritance rules by passing along the sequence generator definition unchanged once the ID column had been defined.
#### Consequences of the change suggested here
This PR reverts the changes made to `@SequenceGeneratorDefinition` inheritance behaviour that were done in #10455.
This means that with the new "report fields where declared" driver mode (which is active in our functional tests) we can not expect the sequence generator definition to be inherited from mapped superclasses. The two test cases from `BasicInheritanceMappingTest` are removed.
I will leave a notice in #10455 to indicate that the new driver mode also affects sequence generator definitions.
The `GH10927Test` test case validates the sequence names generated in a few cases. In fact, I wrote this test against the `2.15.x` branch to make sure we get results that are consistent with the previous behaviour.
This also means `@SequenceGeneratorDefinition` on mapped superclasses is pointless: The mapped superclass does not make use of the definition itself (it has no table), and the setting is never inherited to child classes.
Fixes#10927. There is another implementation with slightly different inheritance semantics in #11052, in case the fix is not good enough and we'd need to review the topic later on.
In order to resolve#10348, some changes were included in #10547 to improve the computed _delete_ order for entities.
One assumption was that foreign key references with `ON DELETE SET NULL` or `... CASCADE` need not need to be taken into consideration when planning the deletion order, since the RDBMS would unset or cascade-delete such associations by itself when necessary. Only associations that do _not_ use RDBMS-level cascade handling would be sequenced, to make sure the referring entity is deleted before the referred-to one.
This assumption is wrong for `ON DELETE CASCADE`. The following examples give reasons why we need to also consider such associations, and in addition, we need to be able to deal with cycles formed by them.
In the following diagrams, `odc` means `ON DELETE CASCADE`, and `ref` is a regular foreign key with no extra `ON DELETE` semantics.
```mermaid
graph LR;
C-->|ref| B;
B-->|odc| A;
```
In this example, C must be removed before B and A. If we ignore the B->A dependency in the delete order computation, the result may not to be correct. ACB is not a working solution.
```mermaid
graph LR;
A-->|odc| B;
B-->|odc| A;
C-->|ref| B;
```
This is the situation in #10912. We have to deal with a cycle in the graph. C must be removed before A as well as B. If we ignore the B->A dependency (e.g. because we set it to "optional" to get away with the cycle), we might end up with an incorrect order ACB.
```mermaid
graph LR;
A-->|odc| B;
B-->|odc| A;
A-->|ref| C;
C-->|ref| B;
```
This example has no possible remove order. But, if we treat `odc` edges as optional, A -> C -> B would wrongly be deemed suitable.
```mermaid
graph LR;
A-->|ref| B;
B-->|odc| C;
C-->|odc| B;
D-->|ref| C;
```
Here, we must first remove A and D in any order; then, B and C in any order. If we treat one of the `odc` edges as optional, we might find the invalid solutions ABDC or DCAB.
#### Solution implemented in this PR
First, build a graph with a node for every to-be-removed entity, and edges for `ON DELETE CASCADE` associations between those entities. Then, use [Tarjan's algorithm](https://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm) to find strongly connected components (SCCs) in this graph. The significance of SCCs is that whenever we remove one of the entities in a SCC from the database (no matter which one), the DBMS will immediately remove _all_ the other entities of that group as well.
For every SCC, pick one (arbitrary) entity from the group to represent all entities of that group.
Then, build a second graph. Again we have nodes for all entities that are to be removed. This time, we insert edges for all regular (foreign key) associations and those with `ON DELETE CASCADE`. `ON DELETE SET NULL` can be left out. The edges are not added between the entities themselves, but between the entities representing the respective SCCs.
Also, for all non-trivial SCCs (those containing more than a single entity), add dependency edges to indicate that all entities of the SCC shall be processed _after_ the entity representing the group. This is to make sure we do not remove a SCC inadvertedly by removing one of its entities too early.
Run a topological sort on the second graph to get the actual delete order. Cycles in this second graph are a problem, there is no delete order.
Fixes#10912.
It will make fuzzy matchers more efficient, and configuration files more readable.
- lib/Doctrine/ORM becomes just src
- tests/Doctrine/ becomes just tests
Spotted while trying to merge https://github.com/doctrine/orm/pull/11076
(among other things) up into 3.0.x. On that branch, it is no longer
possible for an entity to extend another entity without specifying an
inheritance mapping type.
I think the goal of that inheritance was just to reuse the identifier
anyway, so let's just duplicate the identifier declaration instead.
This PR changes a detail in the commit order computation for depended-upon entities.
We have a parent-child relationship between two entity classes. The association is parent one-to-many children, with the child entities containing the (owning side) back-reference.
Cascade-persist is not used, so all entities have to be passed to `EntityManager::persist()`.
Before v2.16.0, two child entities C1 and C2 will be inserted in the same order in which they are passed to `persist()`, and that is regardless of whether the parent entity was passed to `persist()` before or after the child entities.
As of v2.16.0, passing the parent entity to `persist()` _after_ the child entities will lead to an insert order that is _reversed_ compared to the order of `persist()` calls.
This PR makes the order consistent in both cases, as it was before v2.16.0.
#### Cause
When the parent is passed to `persist()` after the children, commit order computation has to re-arrange the entities. The parent must be inserted first since it is referred to by the children.
The implementation of the topological sort from #10547 processed entities in reverse `persist()` order and unshifted finished nodes to an array to obtain the final result. That leads to dependencies (parent → before c1, parent → before c2) showing up in the result in the reverse order of which they were added.
This PR changes the topological sort to produce a result in the opposite order ("all edges pointing left"), which helps to avoid the duplicate array order reversal.
#### Discussion
* This PR _does not_ change semantics of the `persist()` so that entities would (under all ciscumstances) be inserted in the order of `persist()` calls.
* It fixes an unnecessary inconsistency between versions before 2.16.0 and after. In particular, it may be surprising that the insert order for the child entities depends on whether another referred-to entity (the parent) was added before or after them.
* _Both_ orders (c1 before or after c2) are technically and logically correct with regard to the agreement that `commit()` is free to arrange entities in a way that allows for efficient insertion into the database.
Fixes#11058.
Property names as returned by a cast to array are mangled, and that
mangling is not documented. Returning unprefixed produces the same
result, and is more likely to be supported by external tools relying on
the documented possible return values of __sleep.
For instance symfony/var-exporter does not support mangled names, which
leads to issues when caching query parsing results in Symfony
applications.
* derrabus/3.0.x:
Deprecate annotation classes for named queries
Fix typos
Housekeeping: Revert change to AbstractExporter, not needed without subselect fetch.
Address review comments.
Explain internals of eager loading in a bit more detail and how its configured.
1:1 and M:1 associations also use fetch batch size configuration now.
Add another testcase for DQL based fetch eager of collection.
last violation hopefully
Static analysis
Housekeeping: phpcs
Directly load many to many collections, batching not supported yet. fix tests.
Avoid new fetch mode, use this strategy with fetch=EAGER for collections.
Make sure to many assocatinos are also respecting AbstractQuery::setFetchMode
Disallow use of fetch=SUBSELECT on to-one associations.
Go through Persister API instead of indirectly through repository.
Introduce configuration option for subselect batch size.
Houskeeping: phpcs
Disallow WITH keyword on fetch joined associatiosn via subselect.
[GH-1569] Add new SUBSELECT fetch mode for OneToMany associations.
* 2.17.x:
Deprecate annotation classes for named queries
Fix typos
Housekeeping: Revert change to AbstractExporter, not needed without subselect fetch.
Address review comments.
Explain internals of eager loading in a bit more detail and how its configured.
1:1 and M:1 associations also use fetch batch size configuration now.
Add another testcase for DQL based fetch eager of collection.
last violation hopefully
Static analysis
Housekeeping: phpcs
Directly load many to many collections, batching not supported yet. fix tests.
Avoid new fetch mode, use this strategy with fetch=EAGER for collections.
Make sure to many assocatinos are also respecting AbstractQuery::setFetchMode
Disallow use of fetch=SUBSELECT on to-one associations.
Go through Persister API instead of indirectly through repository.
Introduce configuration option for subselect batch size.
Houskeeping: phpcs
Disallow WITH keyword on fetch joined associatiosn via subselect.
[GH-1569] Add new SUBSELECT fetch mode for OneToMany associations.
When unserializing from a cache entry in the previous format, the
sqlStatements need to be copied from the legacy property to the new
property before the reference is created.
The idea here is that instead of having a backward compatibility layer
in the next major branch, we can have a forward compatibility layer in
this branch.
This requires heavily adapting tests, because the proxy instance must:
- be an instance of InternalProxy (easy)
- be a valid entity (hard, especially for PHPUnit)
When transforming these phpdoc types into native types, things break
down. They are correct according to the EBNF, but in practice, there are
so-called phase 2 optimizations that allow using ConditionalPrimary,
ConditionalFactor and ConditionalTerm instances in places where
ConditionalExpression is used.
doctrine/common has been split in several packages. A lot of what was
true about doctrine/common is true about doctrine/persistence today, so
let us simply reuse the existing paragraphs and mention persistence
instead of common.
This reduces our dependency to this shared library that now holds very
little code we use.
The class has not been copied verbatim:
- Unused parameters and methods have been removed.
- The class is final and internal.
- Coding standards have been enforced, including enabling strict_types,
which lead to casting a variable to string before feeding it to
explode().
- A bug found by static analysis has been addressed, where an INI
setting obtained with ini_get() was compared with true, which is never
returned by that function.
- Tests are improved to run on all PHP versions
It is no longer possible to use the "PARTIAL" keyword in a DQL query, or
to artificially build an AST with a partial object expression. It is
still possible to use the result set mapping API to build partial
objects.
What was optimal 10 years ago no longer is, and things might change in
the future. Using AUTO is still the best solution in most cases, and it
should be easy to make it mean something else when it is not.
* 2.17.x:
Allow creating mocks of the Query class (#10990)
Add missing "deprecated" annotation on the annotation driver
Deprecate EntityManager*::getPartialReference()
* 2.17.x:
document Paginator::HINT_ENABLE_DISTINCT
allow to disable "DISTINCT" added to the sql query by the limit subquery walker
Test against php 8.3 (#10963)
update checkout version to version 4
* Use lazy ghosts unconditionally
* Stop extending proxy factory from doctrine/common
Extending it no longer serves any purpose.
* Transform annotation into actual method
It is useful to catch misconfigured dependency constraints. It was
removed in 413c33274d.
This implies configuring mocks so as to support psr/cache 1
psr/cache 1 does not use native return types, and phpdoc is not enough
to obtain a mock that has typed methods.
* 2.16.x:
PHPStan 1.10.35, Psalm 5.15.0 (#10958)
docs: in text, refer to attributes when talking about metadata (#10956)
Fix bullet list layout (#10951)
docs[query-builder]: fix rendering of `Doctrine\DBAL\ParameterType::*` (#10945)
tests[ORMSetupTest]: testCacheNamespaceShouldBeGeneratedForApcu requires enabled apc (#10940)
docs: use modern named arguments syntax
Ignore "Unknown directive" error
Use a stable release
Remove output directory argument
tutorials[getting-started]: example fix bug id type definition
Verify UnitOfWork::HINT_DEFEREAGERLOAD exists and is true
This avoid situations where you might map a decimal to a float, when it
should really be mapped to a string. This is a big pitfall because in
that situation, the ORM will consider the field changed every time.
We have a lot of errors about "Unknown directive" that we should make
known when implementing guides for Doctrine, but cannot address by
modifying the docs.
The unknown directives are:
- configuration-block
- toc
- tocheader
- sectionauthor
* 2.16.x:
Use required classes for Lifecycle Callback examples (#10916)
Add space before backquote (#10918)
Add back throws annotation to getSingleScalarResult (#10907)
Fix link on known issues docs (#10904)
According to the RST docs,
> [inline markup] it must be separated from surrounding text by non-word
> characters. Use a backslash escaped space to work around that: thisis\ *one*\ word.
Because we were missing a space before backquotes here, the links were
not rendered. Escaping the space allow not to actually produce a space
in the output.
See https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#inline-markup
The changes from #10547, which landed in 2.16.0, cause problems for users calling `EntityManager::flush()` from within `postPersist` event listeners.
* When `UnitOfWork::commit()` is re-entered, the "inner" (reentrant) call will start working through all changesets. Eventually, it finishes with all insertions being performed and `UoW::$entityInsertions` being empty. After return, the entity insertion order, an array computed at the beginning of `UnitOfWork::executeInserts()`, still contains entities that now have been processed already. This leads to a strange-looking SQL error where the number of parameters used does not match the number of parameters bound. This has been reported as #10869.
* The fixes made to the commit order computation may lead to a different entity insertion order than previously. `postPersist` listener code may be affected by this when accessing generated IDs for other entities than the one the event has been dispatched for. This ID may not yet be available when the insertion order is different from the one that was used before 2.16. This has been mentioned in https://github.com/doctrine/orm/pull/10906#issuecomment-1682417987.
This PR suggests to address both issues by dispatching the `postPersist` event only after _all_ new entities have their rows inserted into the database. Likewise, dispatch `postRemove` only after _all_ deletions have been executed.
This solves the first issue because the sequence of insertions or deletions has been processed completely _before_ we start calling event listeners. This way, potential changes made by listeners will no longer be relevant.
Regarding the second issue, I think deferring `postPersist` a bit until _all_ entities have been inserted does not violate any promises given, hence is not a BC break. In 2.15, this event was raised after all insertions _for a particular class_ had been processed - so, it was never an "immediate" event for every single entity. #10547 moved the event handling to directly after every single insertion. Now, this PR moves it back a bit to after _all_ insertions.
Fix regression introduced in #10870
`$result = $this->execute(null, $hydrationMode);` in `getSingleResult` can still throw NoResultException exception.
#10880 reports a case where the changes from #10785 cause entity updates to be missed.
Upon closer inspection, this change seems to be causing it:
https://github.com/doctrine/orm/pull/10785/files#diff-55a900494fc8033ab498c53929716caf0aa39d6bdd7058e7d256787a24412ee4L2990-L3003
The code was changed to use `registerManaged()` instead, which basically does the same things, but (since #10785) also includes an additional check against duplicate entity instances.
But, one detail slipped through tests and reviews: `registerManaged()` also updates `\Doctrine\ORM\UnitOfWork::$originalEntityData`, which is used to compute entity changesets. An empty array `[]` was passed for $data here.
This will make the changeset computation assume that a partial object was loaded and effectively ignore all field updates here:
a616914887/lib/Doctrine/ORM/UnitOfWork.php (L762-L764)
I think that, effectively, it is sufficient to call `registerManaged()` only in the two cases where a proxy was created.
Calling `registerManaged()` with `[]` as data for a proxy object is consistent with e. g. `\Doctrine\ORM\EntityManager::getReference()`.
In the case that a full entity has to be loaded, we need not call `registerManaged()` at all, since that will already happen inside `EntityManager::find()` (or, more specifically, `UnitOfWork::createEntity()` called inside it).
Note that the test case has to make some provisions so that we actually reach this case:
* Load an entity that uses `fetch="EAGER"` on a to-one association
* That association being against a class that uses inheritance (why's that?)
* 2.16.x:
Turn identity map collisions from exception to deprecation notice (#10878)
Add possibility to set reportFieldsWhereDeclared to true in ORMSetup (#10865)
Fix UnitOfWork->originalEntityData is missing not-modified collections after computeChangeSet (#9301)
Add an UPGRADE notice about the potential changes in commit order (#10866)
Update branch metadata (#10862)
In #10785, a check was added that prevents entity instances from getting into the identity map when another object for the same ID is already being tracked.
This caused regressions for users that work with application-provided IDs and expect this condition to fail with `UniqueConstraintViolationExceptions` when flushing to the database.
Thus, this PR turns the exception into a deprecation notice. Users can opt-in to the new behavior. In 3.0, the exception will be used.
Implements #10871.
* Fix original data incomplete after flush
* Apply suggestions from code review
Co-authored-by: Alexander M. Turek <me@derrabus.de>
---------
Co-authored-by: Alexander M. Turek <me@derrabus.de>
This excludes such associations from the commit order computation, since the foreign key constraint will be satisfied when inserting the row.
See https://github.com/doctrine/orm/pull/10735/ for more details about this edge case.
Since support for persistence 2 has been dropped, this method may no
longer acces an aliased class name.
Besides, providing an FQCN with a leading backslash should work since
removing it is the first thing that happens inside
AbstractClassMetadataFactory::getMetadataFor().
1. Inherit ClassMetadataInfo::requiresFetchAfterChange flag from root entity when process parent columns mapping (see ClassMetadataInfo::addInheritedFieldMapping(), it uses same condition as ClassMetadataInfo::mapField()) so JoinedSubclassPersister::assignDefaultVersionAndUpsertableValues() to be called in JoinedSubclassPersister::executeInserts().
2. Override JoinedSubclassPersister::fetchVersionAndNotUpsertableValues() to fetch all parent tables (see $this->getJoinSql() call) generated columns. So make protected BasicEntityPersister::identifierFlattener stateless service (use it flattenIdentifier() method) and BasicEntityPersister::extractIdentifierTypes() (to avoid copy-paste).
3. JoinedSubclassPersister::fetchVersionAndNotUpsertableValues() doesnt check empty $columnNames because it would be an error if ClassMetadataInfo::requiresFetchAfterChange is true while no generated columns in inheritance hierarchy.
4. Initialize JoinedInheritanceRoot not-nullable string properties with insertable=false attribute to avoid attempt to insert default null data which cause error:
PDOException: SQLSTATE[23502]: Not null violation: 7 ERROR: null value in column "rootwritablecontent" of relation "joined_inheritance_root" violates not-null constraint
DETAIL: Failing row contains (5, null, dbDefault, dbDefault, , nonUpdatable).
while $rootTableStmt->executeStatement() because JoinedSubclassPersister::getInsertColumnList() have no $insertData (prepared later) to decide is generated column provided by client code or not (so need to skip column)
1. Postgres gives error when insert root entity ($rootTableStmt->executeStatement()) in JoinedSubclassPersister::executeInserts():
PDOException: SQLSTATE[08P01]: <<Unknown error>>: 7 ERROR: bind message supplies 4 parameters, but prepared statement "" requires 6
so exclude notInsertable columns from JoinedSubclassPersister::getInsertColumnList() like it done in parent::prepareInsertData() which call BasicEntityPersister::prepareUpdateData(isInsert: true) where we have condition:
if ($isInsert && isset($fieldMapping['notInsertable']))
2. Try to get generated (notInsertable|notUpdatable) column value on flush() with JoinedSubclassPersister::executeInserts() also fails:
Unexpected empty result for database query.
because method it calls $this->assignDefaultVersionAndUpsertableValues() after insert root entity row, while generated columns in child-entity table, so move call just after insert child row
3. Use option['default'] = 'dbDefault' in functional test entities, to emulate generated value on insert, but declare as generated = 'ALWAYS' for tests purpose (correctness of JoinedSubclassPersister::fetchVersionAndNotUpsertableValues() sql-query)
4. Use JoinedInheritanceRoot::rootField to skip JoinedSubclassPersister::update() optimization for empty changeset in updatable:false columns tests
In 7fa3e6ec7c, a global search and replace
was used for http and https.
This broke the documentation examples in that as soon as you turn on XSD
validation, it will fail because the namespace in the XML file does not
match the ones defined in the XSD file, which do not exhibit the https.
Note that this is not a security concern, because these URIs are
not meant to be actually resolved, but to serve as a unique identifier
for the namespace in which we define our elements.
This change improves scheduling of extra updates in the `BasicEntityPersister`.
Extra updates can be avoided when
* the referred-to entity has already been inserted during the current insert batch/transaction
* we have a self-referencing entity with application-provided ID values (the `NONE` generator strategy).
As a corollary, with this change applications that provide their own IDs can define self-referencing associations as not NULLable.
I am considering this a bugfix since the ORM previously executed additional queries that were not strictly necessary, and that required users to work with NULLable columns where conceptually a non-NULLable column would be valid and more expressive.
One caveat, though:
In the absence of entity-level commit ordering (#10547), it is not guaranteed that entities with self-references (at the class level) will be inserted in a suitable order. The order depends on the sequence in which the entities were added with `persist()`.
Fixes#7877, closes#7882.
Co-authored-by: Sylvain Fabre <sylvain.fabre@assoconnect.com>
The way we have our docs, the sidebar is a separate document and as
such, needs a title. Let us prepend a dummy title until the guides-cli
provides a way to ignore that particular error.
* 2.15.x: (23 commits)
Fix cloning entities when using lazy-ghost proxies
Fixes recomputation of single entity change set when entity contains enum attributes. Due to the fact that originalEntityData contains enum objects and ReflectionEnumProperty::getValue() returns value of enum, comparison of values are always falsy, resulting to update columns value even though it has not changes.
Fix code style issues
Use absolute references
Avoid colon followed by double colon
Use correct syntax for references
Fix invalid reference syntax
Use rst syntax
Use internal link
Escape pipes
Introduce new workflow to test docs
Remove lone dash (#10812)
Treat id field proprites same as regular field
Move three "Ticket/"-style tests to the right namespace
Follow recommendation about multiline type
Fix unserialize() errors when running tests on PHP 8.3 (#10803)
Explain `EntityManager::getReference()` peculiarities (#10800)
Upgrade to Psalm 5.13
test: assert `postLoad` has data first
distinct() updates QueryBuilder state correctly
...
Due to the fact that originalEntityData contains enum objects and
ReflectionEnumProperty::getValue() returns value of enum,
comparison of values are always falsy, resulting to update
columns value even though it has not changes.
According to the Sphinx docs, when in reference/architecture.rst, a
reference to reference/inheritance-mapping would resolve to
reference/reference/inheritance-mapping.rst, because it is relative to
the current document
There are a few requests (#5742, #5368, #5109, #6776) that ask to change the order of operations in the UnitOfWork to perform "deletes before inserts", or where such a switch appears to solve a reported problem.
I don't want to say that this is not doable. But this PR at least adds two tricky examples where INSERTs need to be done before an UPDATE can refer to new database rows; and where the UPDATE needs to happen to release foreign key references to other entities before those can be DELETEd.
So, at least as long as all operations of a certain type are to be executed in blocks, this example allows no other order of operations than the current one.
While trying to understand #3037, I found that it may happen that we have more entries in `\Doctrine\ORM\UnitOfWork::$entityIdentifiers` than in `\Doctrine\ORM\UnitOfWork::$identityMap`.
The former is a mapping from `spl_object_id` values to ID hashes, the latter an array first of entity class names and then from ID hash to entity object instances.
(Basically, "ID hash" is a concatenation of all field values making up the `@Id` for a given entity.)
This means that at some point, we must have _different_ objects representing the same entity, or at least over time different objects are used for the same entity without the UoW properly updating its `::$entityIdentifiers` structure.
I don't think it makes sense to overwrite an entity in the identity map, since that means a currently `MANAGED` entity is replaced with something else.
If it makes sense at all to _replace_ an entity, that should happen through dedicated management methods to first detach the old entity before persisting, merging or otherwise adding the new one. This way we could make sure the internal structures remain consistent.
Apparently, there is consensus about multiline types between:
- PHPStan
- Psalm
- Slevomat Coding Standard
See https://github.com/slevomat/coding-standard/issues/1586#issuecomment-1610195706
Using parenthesis is less ambiguous, it makes it clear to the parser
where the type begins and where it ends.
The change has a positive impact on the Psalm baseline, showing
that psalm-return annotation was not really understood previously.
`EntityManager::merge()` has been deprecated in #8461 and removed in #9488.
This PR removes a few remaining references and artefacts that - to my understanding - refer to it.
A lot of our tests mention it, but I do not think it is important to the
test. Maybe it was a way to have more efficient tests? Most times,
removing the hint does not affect the test outcome.
* Explain `EntityManager::getReference()` peculiarities
As one takeaway from https://github.com/doctrine/orm/issues/3037#issuecomment-1605657003 and #843, we should look into better explaining the `EntityManager::getReference()` method, it’s semantics, caveats and potential responsibilities placed on the user.
This PR tries to do that, so it fixes#10797.
* Update docs/en/reference/advanced-configuration.rst
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
---------
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
Previously calling distinct() when the QueryBuilder was in clean state would cause subsequent getDQL() calls to ignore the distinct queryPart
Fixes#10784
This PR tries to improve the situation/problem explained in #3037:
Under certain conditions – there may be multiple and not all are known/well-understood – we may get inconsistencies between the `\Doctrine\ORM\UnitOfWork::$entityIdentifiers` and `\Doctrine\ORM\UnitOfWork::$identityMap` arrays.
Since the `::$identityMap` is a plain array holding object references, objects contained in it cannot be garbage-collected.
`::$entityIdentifiers`, however, is indexed by `spl_object_id` values. When those objects are destructed and/or garbage-collected, the OID may be reused and reassigned to other objects later on.
When the OID re-assignment happens to be for another entity, the UoW may assume incorrect entity states and, for example, miss INSERT or UPDATE operations.
One cause for such inconsistencies is _replacing_ identity map entries with other object instances: This makes it possible that the old object becomes GC'd, while its OID is not cleaned up. Since that is not a use case we need to support (IMHO), #10785 is about adding a safeguard against it.
In this test shown here, the `merge()` operation is currently too eager in creating a proxy object for another referred-to entity. This proxy represents an entity already present in the identity map at that time, potentially leading to this problem later on.
Add tests for entity insertion and deletion that require the commit order calculation to happen on the entity level. This demonstrates the necessity for the changes in #10547.
This PR contains two tests with carefully constructed entity relationships, where we have a non-nullable `parent` foreign key relationships between entities stored in the same table.
Class diagram:
```mermaid
classDiagram
direction LR
class A
class B
A --> B : parent
B --|> A
```
Object graph:
```mermaid
graph LR;
b1 --> b2;
b2 --> a;
b3 --> b2;
```
#### Situation before #10547
The commit order is computed by looking at the associations at the _table_ (= _class_) level. Once the ordering of _tables_ has been found, entities being mapped to the same table will be processed in the order they were given to `persist()` or `remove()`.
That means only a particular ordering of `persist()` or `remove()` calls (see comment in the test) works:
For inserts, the order must be `$a, $b2, $b1, $b3` (or `... $b3, $b1`), for deletions `$b1, $b3, $b2, $a`.
#### Situation with entity-level commit order computation (as in #10547)
The ORM computes the commit order by considering associations at the _entity_ level. It will be able to find a working order by itself.
This refactoring does two things:
* We can avoid collecting the post insert IDs in a cumbersome array structure that will be returned by the EntityPersisters and processed by the UoW right after. Instead, use a more expressive API: Make the EntityPersisters tell the UoW about the IDs immediately.
* IDs will be available in inserted entities a tad sooner. That may help to resolve#10735, where we can use the IDs to skip extra updates.
When collection updates/join table cleanups do not happen through specialized Entity-/CollectionPersister methods but instead as "plain" updates, we may issue a lot more queries than expected.
In my case a custom doctrine type of Uuid object is converted to string by simply casting it, resulting in a hex DELETE FROM x WHERE id = ? query,
whilst it should've been converted along the way to it's binary representation. This leads to no deletions being made at all as you would expect making use of doctrine custom type's as an identifier.
This commit fixes usage of ramsey/uuid or symfony/uid as custom id types when making use of orphan removal.
This is the result of the contradiction between the phpdoc
(ClassMetadata), and the condition, which guarantees $metadata is not a
ClassMetadata. Relaxing the phpdoc leads to other phpstan issues, about
properties that exist in ClassMetadata but not in
PersistentClassMetadata. The right way to fix this would be to switch
from a deprecation to an exception, but that is not the path we have
taken, and all this will disappear in 3.0.x anyway, so let's not bother.
Add tests for entity insertion and deletion that require writes to different tables in an interleaved fashion, and that have to re-visit a particular table.
#### Background
In #10531, I've given an example where it is necessary to compute the commit order on the entity (instead of table) level.
Taking a closer look at the UoW to see how this could be achieved, I noticed that the current, table-level commit order manifests itself also in the API between the UoW and `EntityPersister`s.
#### Current situation
The UoW computes the commit order on the table level. All entity insertions for a particular table are passed through `EntityPersister::addInsert()` and finally written through `EntityPersister::executeInserts()`.
#### Suggested change
The test in this PR contains a carefully constructed set of four entities. Two of them are of the same class (are written to the same table), but require other entities to be processed first.
In order to be able to insert this set of entities, the ORM must be able to perform inserts for a given table repeatedly, interleaved with writing other entities to their respective tables.
This is part of the series of issues fixed by #10547. In particular, the changes from #10566 were relevant.
See #10348 for the bug description.
Co-authored-by: Grégoire Paris <postmaster@greg0ire.fr>
This is the third step to break https://github.com/doctrine/orm/pull/10547 into smaller PRs suitable for reviewing. It uses the new topological sort implementation from #10592 and the refactoring from #10651 to compute the UoW's commit order for entity insertions not on the entity class level, but for single entities and their actual dependencies instead.
#### Current situation
`UnitOfWork::getCommitOrder()` would compute the entity sequence on the class level with the following code:
70477d81e9/lib/Doctrine/ORM/UnitOfWork.php (L1310-L1325)
#### Suggested change
* Instead of considering the classes of all entities that need to be inserted, updated or deleted, consider the new (inserted) entities only. We only need to find a sequence in situations where there are foreign key relationships between two _new_ entities.
* In the dependency graph, add edges for all to-one association target entities.
* Make edges "optional" when the association is nullable.
#### Test changes
I have not tried to fully understand the few changes necessary to fix the tests. My guess is that those are edge cases where the insert order changed and we need to consider this during clean-up.
Keep in mind that many of the functional tests we have assume that entities have IDs assigned in the order that they were added to the EntityManager. That does not change – so the order of entities is generally stable, equal to the previous implementation.
When computing the commit order for entity removals, we have to look out for `@ORM\JoinColumn(onDelete="SET NULL")` to find places where cyclic associations can be broken.
#### Background
The UoW computes a "commit order" to find the sequence in which tables shall be processed when inserting entities into the database or performing delete operations.
For the insert case, the ORM is able to schedule _extra updates_ that will be performed after all entities have been inserted. Associations which are configured as `@ORM\JoinColumn(nullable=true, ...)` can be left as `NULL` in the database when performing the initial `INSERT` statements, and will be updated once all new entities have been written to the database. This can be used to break cyclic associations between entity instances.
For removals, the ORM does not currently implement up-front `UPDATE` statements to `NULL` out associations before `DELETE` statements are executed. That means when associations form a cycle, users have to configure `@ORM\JoinColumn(onDelete="SET NULL", ...)` on one of the associations involved. This transfers responsibility to the DBMS to break the cycle at that place.
_But_, we still have to perform the delete statements in an order that makes this happen early enough. This may be a _different_ order than the one required for the insert case. We can find it _only_ by looking at the `onDelete` behaviour. We must ignore the `nullable` property, which is irrelevant, since we do not even try to `NULL` anything.
#### Example
Assume three entity classes `A`, `B`, `C`. There are unidirectional one-to-one associations `A -> B`, `B -> C`, `C -> A`. All those associations are `nullable= true`.
Three entities `$a`, `$b`, `$c` are created from these respective classes and associations are set up.
All operations `cascade` at the ORM level. So we can test what happens when we start the operations at the three individual entities, but in the end, they will always involve all three of them.
_Any_ insert order will work, so the improvements necessary to solve #10531 or #10532 are not needed here. Since all associations are between different tables, the current table-level computation is good enough.
For the removal case, only the `A -> B` association has `onDelete="SET NULL"`. So, the only possible execution order is `$b`, `$c`, `$a`. We have to find that regardless of where we start the cascade operation.
The DBMS will set the `A -> B` association on `$a` to `NULL` when we remove `$b`. We can then remove `$c` since it is no longer being referred to, then `$a`.
#### Related cases
These cases ask for the ORM to perform the extra update before the delete by itself, without DBMS-level support:
* #5665
* #10548
mappedBy is never defined on an inverse relationship.
Bi-directional self-referencing should IMO still result in 2 separate
associations, on 2 separate fields of the same class, like mentor or
mentee.
Maybe we do not know enough about the parameter to determine the type of
the returned relationship, but we can at least narrow it down to 3
possibilites.
These methods assert the type of the mapping provided by the collection
according to the name of the class they are in: the one to many
persister only ever deals with one to many associations, and the many to
many persister only ever deals with many to many associations.
Throughout the codebase, there is this pattern where we ensure we have
the owning side of an association.
It involves accessing it from the associationMappings array. In the end,
static analysis cannot know that the association is indeed owning.
By introducing this convenience method, we make this clear, and also
delegate the complexity to the class metadata factory.
Interfaces cannot have properties, and we do not have a concept of
sealed classes available to us without installing third party packages.
Interfaces can have methods however, which allows us to simplify calling
code.
I've been avoiding introducing getters for mapping properties because I
do not know what the performance implications are, but here, I think it
is sensible to make an exception, given the benefits.
This is the first chunk to break #10547 into smaller PRs suitable for reviewing. It adds a new topological sort implementation.
#### Background
Topological sort is an algorithm that sorts the vertices of a directed acyclic graph (DAG) in a linear order such that for every directed edge from vertex A to vertex B, vertex A comes before vertex B in the ordering. This ordering is called a topological order.
Ultimately (beyond the scope of this PR), in the ORM we'll need this to find an order in which we can insert new entities into the database. When one entity needs to refer to another one by means of a foreign key, the referred-to entity must be inserted before the referring entity. Deleting entities is similar.
A topological sorting can be obtained by running a depth first search (DFS) on the graph. The order in which the DFS finishes on the vertices is a topological order. The DFS is possible iif there are no cycles in the graph. When there are cycles, the DFS will find them.
For more information about topological sorting, as well as a description of an DFS-based topological sorting algorithm, see https://en.wikipedia.org/wiki/Topological_sorting.
#### Current situation
There is a DFS-based topological sorting implemented in the `CommitOrderCalculator`. This implementation has two kinks:
1. It does not detect cycles
When there is a cycle in the DAG that cannot be resolved, we need to know about it. Ultimately, this means we will not be able to insert entities into the database in any order that allows all foreign keys constraints to be satisfied.
If you look at `CommitOrderCalculator`, you'll see that there is no code dealing with this situation.
2. It has an obscure concept of edge "weights"
To me, it is not clear how those are supposed to work. The weights are related to whether a foreign key is nullable or not, but can (could) be arbitrary integers. An edge will be ignored if it has a higher (lower) weight than another, already processed edge... 🤷🏻?
#### Suggested change
In fact, when inserting entities into the database, we have two kinds of foreign key relationships: Those that are `nullable`, and those that are not.
Non-nullable foreign keys are hard requirements: Referred-to entities must be inserted first, no matter what. These are "non-optional" edges in the dependency graph.
Nullable foreign keys can be set to `NULL` when first inserting an entity, and then coming back and updating the foreign key value after the referred-to (related) entity has been inserted into the database. This is already implemented in `\Doctrine\ORM\UnitOfWork::scheduleExtraUpdate`, at the expense of performing one extra `UPDATE` query after all the `INSERT`s have been processed. These edges are "optional".
When finding a cycle that consists of non-optional edges only, treat it as a failure. We won't be able to insert entities with a circular dependency when all foreign keys are non-NULLable.
When a cycle contains at least one optional edge, we can use it to break the cycle: Use backtracking to go back to the point before traversing the last _optional_ edge. This omits the edge from the topological sort order, but will cost one extra UPDATE later on.
To make the transition easier, the new implementation is provided as a separate class, which is marked as `@internal`.
#### Outlook
Backtracking to the last optional edge is the simplest solution for now. In general, it might be better to find _another_ (optional) edge that would also break the cycle, if that edge is also part of _other_ cycles.
Remember, every optional edge skipped costs an extra UPDATE query later on. The underlying problem is known as the "minimum feedback arc set" problem, and is not easily/efficiently solvable. Thus, for the time being, picking the nearest optional edge seems to be reasonable.
This PR will make the annotations and attribute mapping drivers report mapping configuration for the classes where it is declared, instead of omitting it and reporting it for subclasses only. This is necessary to be able to catch mis-configurations in `ClassMetadataFactory`.
Fixes#10417, closes#10450, closes#10454.
#### ⚠️ Summary for users getting `MappingExceptions` with the new mode
When you set the `$reportFieldsWhereDeclared` constructor parameters to `true` for the AnnotationDriver and/or AttributesDriver and get `MappingExceptions`, you may be doing one of the following:
* Using `private` fields with the same name in different classes of an entity inheritance hierarchy (see #10450)
* Redeclaring/overwriting mapped properties inherited from mapped superclasses and/or other entities (see #10454)
As explained in these two PRs, the ORM cannot (or at least, was not designed to) support such configurations. Unfortunately, due to the old – now deprecated – driver behaviour, the misconfigurations could not be detected, and due to previously missing tests, this in turn was not noticed.
#### Current situation
The annotations mapping driver has the following condition to skip properties that are reported by the PHP reflection API:
69c7791ba2/lib/Doctrine/ORM/Mapping/Driver/AnnotationDriver.php (L345-L357)
This code has been there basically unchanged since the initial 2.0 release. The same condition can be found in the attribute driver, probably it has been copied when attributes were added.
I _think_ what the driver tries to do here is to deal with the fact that Reflection will also report `public`/`protected` properties inherited from parent classes. This is supported by the observation (see #5744) that e. g. YAML and XML drivers do not contain this logic.
The conditions are not precise enough for edge cases. They lead to some fields and configuration values not even being reported by the driver.
Only since the fields would be "discovered" again when reflecting on subclasses, they eventually end up in class metadata structures for the subclasses. In one case of inherited ID generator mappings, the `ClassMetadataFactory` would also rely on this behaviour.
Two potential bugs that can result from this are demonstrated in #10450 and #10454.
#### Suggested solution
In order to find a more reliable way of separating properties that are merely reported again in subclasses from those that are actual re-declarations, use the information already available in `ClassMetadata`. In particular, `declared` tells us in which non-transient class a "field" was first seen.
Make the mapping driver skip only those properties for which we already know that they have been declared in parent classes, and skip them only when the observed declaring class matches the expectation.
For all other properties, report them to `ClassMetadataFactory` and let that deal with consistency checking/error handling.
#10450 and #10454 are merged into this PR to show that they pass now.
#### Soft deprecation strategy
To avoid throwing new/surprising exceptions (even for misconfigurations) during a minor version upgrade, the new driver mode is opt-in.
Users will have to set the `$reportFieldsWhereDeclared` constructor parameters to `true` for the `AnnotationDriver` and/or `AttributesDriver`. Unless they do so, a deprecation warning will be raised.
In 3.0, the "new" mode will become the default. The constructor parameter can be deprecated (as of ORM 3.1, probably) and is a no-op.
We need to follow up in other places (DoctrineBundle, ... – what else?) to make this driver parameter an easy-to-change configuration setting.
- Each type is now either final, abstract or an interface.
- The mappedBy attribute is no longer nullable and moved down the
hierarchy.
- The inversedBy attribute is still nullable and also moved down the
hierarchy.
- Code common to ManyToOneAssociationMapping and
OneToOneOwningSideMapping is de-duplicated and moved up the hierarchy
- Code inside ToManyInverseSideMapping and ToManyOwningSideMapping comes
from a trait to avoid duplication.
Instead of ensuring every mapping array has a mappedBy and an inversedBy
field, let us do the opposite, and remove them when they are null.
Likewise if there is a joinColumns field, it is useless if null or
empty.
In the code examples of the use of the ``Query#toIterable()`` method, an index ``$i`` is used to control the size of the current batch, which ends when the condition ``($i % $batchSize) === 0`` becomes true.
Because this condition is preceded by ``++$i`` and ``$i`` is initialized to 1, the first batch is actually of size ``$batchSize - 1`` instead of ``$batchSize``.
To solve this, ``$i`` should be initialized to 0 instead of 1.
Currently, the AttributeDriver ignores any join column attribute specified on a many to many relationship.
Let's copy code from the AnnotationDriver to fix that.
Fixes#9902
Following up on #10652:
#### Current situation
The implementation of `\Doctrine\ORM\Mapping\ClassMetadataInfo::_validateAndCompleteOneToOneMapping` will consider a field with a one-to-one association to be the owning side also when it configures `@JoinColumn` settings.
#### Suggested change
For a one to one association, a field should be the inverse side when it uses the `mappedBy` attribute, and be the owning side otherwise. The `JoinColumn` may be configured on the owning side only.
This PR adds a deprecation notice when `@JoinColumn` is used on the side of a one-to-one association where `mappedBy` occurs.
In 3.0, this will throw a `MappingException`.
This makes no sense because it describes a one-to-one where each table
references the other one. I do not think this is deliberately supported,
and it probably will not be supported at all in 3.0.
This is the second chunk to break #10547 into smaller PRs suitable for reviewing. It prepares the `UnitOfWork` to work with a commit order computed on the entity level, as opposed to a class-based ordering as before.
#### Background
#10531 and #10532 show that it is not always possible to run `UnitOfWork::commit()` with a commit order given in terms of entity _classes_. Instead it is necessary to process entities in an order computed on the _object_ level. That may include
* a particular order for multiple entities of the _same_ class
* a particular order for entities of _different_ classes, possibly even going back and forth (entity `$a1` of class `A`, then `$b` of class `B`, then `$a2` of class `A` – revisiting that class).
This PR is about preparing the `UnitOfWork` so that its methods will be able to perform inserts and deletions on that level of granularity. Subsequent PRs will deal with implementing that particular order computation.
#### Suggested change
Change the private `executeInserts` and `executeDeletions` methods so that they do not take a `ClassMetadata` identifying the class of entities that shall be processed, but pass them the list of entities directly.
The lists of entities are built in two dedicated methods. That happens basically as previously, by iterating over `$this->entityInsertions` or `$this->entityDeletions` and filtering those by class.
#### Potential (BC breaking?) changes that need review scrutiny
* `\Doctrine\ORM\Persisters\Entity\EntityPersister::addInsert()` was previously called multiple times, before the insert would be performed by a call to `\Doctrine\ORM\Persisters\Entity\EntityPersister::executeInserts()`. With the changes here, this batching effectively no longer takes place. `executeInserts()` will always see one entity/insert only, and there may be multiple `executeInserts()` calls during a single `UoW::commit()` phase.
* The caching in `\Doctrine\ORM\Cache\Persister\Entity\AbstractEntityPersister` previously would cache entities from the last executed insert batch only. Now it will collect entities across multiple batches. I don't know if that poses a problem.
* Overhead in `\Doctrine\ORM\Persisters\Entity\BasicEntityPersister::executeInserts` is incurred multiple times; that may, however, only be about SQL statement preparation and might be salvageable.
* The `postPersist` event previously was not dispatched before _all_ instances of a particular entity class had been written to the database. Now, it will be dispatched immediately after every single entity that has been inserted.
During a recent refactoring, I had to pick a relationship type for this
piece of code, and I chose wrong, because a many-to-one cannot have a
mappedBy field.
Since PHPUnit 10, it is possible to display details when notices and
warnings happen, and to fail the test suite on notice.
failOnWarning is older than that.
These properties only make sense for the owning side of a many-to-many
relationship.
Moving them down allows us simplify the serialization code, because the
case when these properties are empty no longer happen.
This makes the array access implementation of FieldMapping useful only
to consumers, it is no longer useful internally, and should be
deprecated as of Doctrine 3.1.0
* Add test case for https://github.com/doctrine/orm/issues/7717
* Do not hide null equality checks in `SqlValueVisitor::walkComparison`
* Annotate `GH7717Parent::$children` type
When an entity have a backed enum as identifier, `UnitOfWork` tries to
cast to string when generating the hash of the id.
This fix calls `->value` when identifier is a `BackedEnum`.
Fixes#10471Fixes#10334
In the past, it has been decided to use arrays for this out of
legitimate performance concerns. But PHP has evolved, and now, it is
more performant and memory efficient to use objects.
DiscriminatorColumnMapping is just a specialization of the array shape
that is right of the pipe: it has the same fields, except fewer fields
are nullable. The union of that is the same thing as the array shape.
This means that a class implementing a doctrine/persistence interface or
extending a class from that package can throw a specialized exception
that will still be caught by code in the parent package.
The default allocationSize for @SequenceGenerator has actually been set to 1 ever since 434325e (back in 2010!)
This was done to remove confusion, but the docs were never updated to reflect the change
This puts the remarks that apply to both JTI/STI in a common section at the beginning, and tries to explain a bit more in detail why it is that way.
It was sparked off by the discussion in #10538.
* 2.15.x:
Skip test instead of commenting it out (#10560)
Add missing return statements to Command:configure methods
Fix a Markdown/RST formatting glitch
Add mapping configurations for classes that were used in tests as entities, but never declared
Allow to-many associations on mapped superclasses w/ ResolveTargetEntityListener
* 2.15.x:
Ignore the cache dir of PHPUnit 10 (#10546)
Make data providers static (#10545)
Make data providers static (#10544)
Bump dev tools (#10541)
Mark SqlWalker methods as not deprecated (#10540)
docs: consistency order for docblock in association mapping (#10534)
Correct use of PHP attribute
fix typo in faq.rst (#10526)
fix: use executeStatement in SchemaTool (#10516)
Write a test in a more specific way
Put up a warning sign that mapping may not be inherited from transient classes (#10392)
Avoid unnecessary information in query hints to improve query cache hit ratio
* 2.14.x:
Ignore the cache dir of PHPUnit 10 (#10546)
Make data providers static (#10544)
Bump dev tools (#10541)
Mark SqlWalker methods as not deprecated (#10540)
docs: consistency order for docblock in association mapping (#10534)
Correct use of PHP attribute
fix typo in faq.rst (#10526)
fix: use executeStatement in SchemaTool (#10516)
Write a test in a more specific way
Put up a warning sign that mapping may not be inherited from transient classes (#10392)
Avoid unnecessary information in query hints to improve query cache hit ratio
phpstan treats implementations of deprecated methods of an interface as being deprecated themselves by default.
However, SqlWalker does not intend to deprecate all those methods that are deprecated in TreeWalker, as they are
moved down. Marking them as not deprecated will avoid reporting usages of deprecated APIs when implementing
custom DQL functions for instance.
Working on converting these array shapes to DTO allowed me to find every
signature where they are supposed to be used.
The Psalm baseline gets worse because it considers accessing an array
key differently depending on whether it is defined vaguely, as
array<string, mixed>, or precisely, as array{my-key?: string}.
This _seems_ to work, but...
To my understanding, that is only because `ReflectionClass::getProperties` will report `protected` properties also for subclasses, including DocBlocks etc. The mapping drivers are unable to tell where a field comes from, so they pick up the mapping and treat it as originating from the inheriting class.
If that is indeed outside of what the ORM was designed for (sombody please confirm?), then we should explicitly discourage it.
Yes, I have seen examples of this in the wild.
Allow to-many associations to be used on mapped superclasses when the owning (inverse) side does not refer back to the mapped superclass, thanks to `ResolveTargetEntityListener`.
#### Current situation
The [documentation states](https://www.doctrine-project.org/projects/doctrine-orm/en/latest/reference/inheritance-mapping.html):
> No database table will be created for a mapped superclass itself
> [...] persistent relationships defined by a mapped superclass must be unidirectional (with an owning side only). This means that One-To-Many associations are not possible on a mapped superclass at all.
That's a though limitation.
~Obviously~ ~apparently~ Probably the limitation comes from the fact that in a to-many association the "many" side has to hold a foreign key. Since the mapped superclass does not have a database table (it's not an entity), no such backreference can be established.
Currently, to-many associations trigger an exception as soon as they are seen on a mapped superclass:
d6c0031d44/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php (L459-L461)
#### `ResolveTargetEntityListener`
The `ResolveTargetEntityListener` can be used to substitute interface or class names in mapping configuration at runtime, during the metadata load phase.
When this gimmick is used to replace _all_ references to the mapped superclass with an entity class in time, it should be possible to have to-many associations on the inheriting entity classes.
#### Suggested solution
Instead of rejecting to-many associations on mapped superclasses right away, validate that at the end of the day (after the `loadClassMetadata` event has been processed) no association may target at a non-entity class. That includes mapped superclasses as well as transient classes.
#### Motivating example
Consider a library that comes with a `User` base class. This class is `abstract` and has to be subclassed/filled when the library is used.
By making this a mapped superclass, library users have the freedom to either have a simple user entity class or a user class hierarchy, but we do not impose any requirements on them. (NB we also don't want to have a root entity in the library, because that would have to declare the entire class hierarchy, including library users' classes.)
The actual user class to be used will be configured through the `ResolveTargetEntityListener`.
The library also includes a `SocialMediaAccount` entity. A `User` can have multiple of these accounts, and we want to be able to navigate the accounts from the user side.
To make the example even more fancy, there is a self-referencing association on the `User`: A `User` has been created by another user, and holds a collection of all other `User`s it created.
The test case contained in this PR contains this example and validates that all association mappings look just as if the final user class had been written as an entity directly, without the superclass.
#### Potential review talking points
- Am I missing other reasons why to-many is not feasible?
- We now reject association mappings with `targetEntity`s that are not entities; relevant BC break? (IMHO: no.)
#### Review tip
Review commit by commit, not all files at once. The last commit adds a lot of entity declarations that were previously missed in tests and now raised exceptions; that's a lot of clutter in the PR.
I've noticed that over time my query caches fill up with redundant queries, i. e. different cache entries for the DQL -> SQL translation that are exactly the same. For me, it's an issue because the cache entries fill up precious OPcache memory.
Further investigation revealed that the queries themselves do not differ, but only the query hints – that are part of the computed cache key – do.
In particular, only the value for the `WhereInWalker::HINT_PAGINATOR_ID_COUNT` query hint are different. Since `WhereInWalker` only needs to know _if_ there are matching IDs but not _how many_, we could avoid such cache misses by using just a boolean value as cache hint.
One could interpret the old description as if `Connection#transactional()` would not rollback the transaction. Also, the fact that the `EntityManager` gets closed in case of an exception was not mentioned.
* 2.14.x:
Remove calls to assertObjectHasAttribute() (#10502)
Remove calls to withConsecutive() (#10501)
Use recognized array key
Fix#9095 by re-applying #9096
Use linebreaks
"joinColumn" has no meaning for the static PHP driver. That's because it
performs no translation, unlike other drivers: we're using the
ClassMetadata API directly.
Before this change, changing the value of the "name" key did not break
the test suite. After this change, it does.
Make the `Paginator`-internal query (`... WHERE ... IN (id, id2,
id3...)`) cacheable in the query cache again.
When the Paginator creates the internal subquery that does the actual
result limiting, it has to take DBAL type conversions for the identifier
column of the paginated root entity into account (#7820, fixed in
#7821).
In order to perform this type conversion, we need to know the DBAL type
class for the root entity's `#[Id]`, and we have to figure it out based
on a given (arbitrary) DQL query. This requires DQL parsing and
inspecting the AST, so #7821 placed the conversion code in the
`WhereInWalker` where all the necessary information is available.
The problem is that type conversion has to happen every time the
paginator is run, but the query that results from running
`WhereInWalker` would be kept in the query cache. This was reported in
#7837 and fixed by #7865, by making this particular query expire every
time. The query must not be cached, since the necessary ID type
conversion happens as a side-effect of running the `WhereInWalker`.
The Paginator internal query that uses `WhereInWalker` has its DQL
re-parsed and transformed in every request.
This PR moves the code that determines the DBAL type out of
`WhereInWalker` into a dedicated SQL walker class, `RootTypeWalker`.
`RootTypeWalker` uses a ~hack~ clever trick to report the type back: It
sets the type as the resulting "SQL" string. The benefit is that
`RootTypeWalker` results can be cached in the query cache themselves.
Only the first time a given DQL query has to be paginated, we need to
run this walker to find out the root entity's ID type. After that, the
type will be returned from the query cache.
With the type information being provided, `Paginator` can take care of
the necessary conversions by itself. This happens every time the
Paginator is used.
The internal query that uses `WhereInWalker` can be cached again since
it no longer has side effects.
Since #10411 has been merged, no more need to specify all intermediate
abstract entity classes.
Thus, we can relax the schema validator check as requested in #9095. The
reasons given in #9142 no longer apply.
Fixes#9095
This spares us from referencing ClassMetadataInfo from other classes,
which is a good thing since it is deprecated. It also means merging up
is easier.
In #10431, some invalid inheritance declarations in our test base were fixed. However, the change missed to update XML, PHP and static PHP mapping configurations as well.
Apparently, this did not raise any flags because the misconfiguration only caused a deprecation notice in 2.15.x. Running the tests against 3.0 (where the misconfiguration will be an error) unveiled the mistake.
Inheritance has to be declared as soon as one entity class extends (directly or through middle classes) another one.
This is also pointed out in the opening comment for #8348:
> Entities are not allowed to extend from entities without an inheritence mapping relationship (Single Table or Joined Table inheritance). [...] While Doctrine so far allowed these things, they are fragile and will break on certain scenarios.
Throwing an exception in case of this misconfiguration is nothing we should do light-heartedly, given that it may surprise users in a bugfix or feature release. So, we should start with a deprecation notice and make this an exception in 3.0. The documentation is updated accordingly at #10429.
Catching missing inheritance declarations early on is important to avoid weird errors further down the road, giving users a clear indication of the root cause.
In case you are affected by this, please understand that although things "previously worked" for you, you have been using the ORM outside of what it was designed to do. That may have worked in simple cases, but may also have caused invalid results (wrong or missing data after hydration?) that possibly went unnoticed in subtle cases.
This picks the test case from #9517 and rebases it onto 2.14.x.
The problem has been covered in #8415, so this PR closes#9517 and fixes#9516.
Co-authored-by: Robert D'Ercole <bobdercole@gmail.com>
This PR prevents the Paginator from causing OpCache "wasted memory" to increase _on every request_ when used with Symfony's `PhpFilesAdapter` as the cache implementation for the query cache.
Depending on configured thresholds, wasted memory this will either cause periodic opcache restarts or running out of memory and not being able to cache additional scripts ([Details](https://tideways.com/profiler/blog/fine-tune-your-opcache-configuration-to-avoid-caching-suprises)).
Fixes#9917, closes#10095.
There is a long story (#7820, #7821, #7837, #7865) behind how the Paginator can take care of DBAL type conversions when creating the pagination query. This conversion has to transform identifier values before they will be used as a query parameter, so it has to happen every time the Paginator is used.
For reasons, this conversion happens inside `WhereInWalker`. Tree walkers like this are used only during the DQL parsing/AST processing steps. Having a DQL query in the query cache short-cuts this step by fetching the parsing/processing result from the cache.
So, to make sure the conversion happens also with the query cache being enabled, this line
1753d03500/lib/Doctrine/ORM/Tools/Pagination/Paginator.php (L165)
was added in #7837. It causes `\Doctrine\ORM\Query::parse()` to re-parse the query every time, but will also put the result into the query cache afterwards.
At this point, the setup described in #9917 – which, to my knowledge, is the default in Symfony + DoctrineBundle projects – will ultimately bring us to this code:
4b3391725f/src/Symfony/Component/Cache/Adapter/PhpFilesAdapter.php (L248-L249)
When writing a cache item with an already existing key, the driver has to make sure the opcache will honor the changed PHP file. This is what causes _wasted memory_ to increase.
Instead of using `\Doctrine\ORM\Query::expireQueryCache()`, which will force `\Doctrine\ORM\Query::parse()` to parse the query again before putting it into the cache, use `\Doctrine\ORM\Query::useQueryCache(false)`. The subtle difference is the latter will not place the processed query in the cache in the first place.
A test case is added to check that repeated use of the paginator does not call the cache to update existing keys. That should suffice to make sure we're not running into the issue, while at the same time not complicating tests by using the `PhpFilesAdapter` directly.
Note that in order to observe the described issue in tests, you will need to use the `PhpFilesDriver` and also make sure that OpCache is enabled and also activated for `php-cli` (which is running the unit tests).
This particular subquery generated/used by the Paginator is not put into the query cache. The DQL parsing/to-SQL conversion has to happen _every time_ the Paginator is used.
This, however, was already the case before this PR. In other words, this PR only changes that we do not store/update the cached result every time, but instead completely omit caching the query.
This should be replaced with a DTO in the next major.
To be able to have something usable, I had to move the validation of the
DTO (checking that it has a "class" attribute) to mapEmbedded, which
happens earlier than doLoadMetadata(). It is unclear to me why it was
not put here in the first place.
* Fixup GH8127 test case
This removes the unnecessary "middle2" class and puts back in the effect of the data provider.
Both changes were accidentally committed when I was working on #10411 and just meant as experiments during debugging.
* Fix CS
Looking at usages in the codebase, it is supposed to be at least
bool|string, but not string.
When dumping the value inside `getFieldMapping`, it is always a boolean.
This fixes two closely related bugs.
1. When inheriting a to-one association from a mapped superclass, update the `sourceEntity` class name to the current class only when the association is actually _declared_ in the mapped superclass.
2. Reject association types that are not allowed on mapped superclasses only when they are actually _declared_ in a mapped superclass, not when inherited from parent classes.
Currently, when a many-to-one association is inherited from a `MappedSuperclass`, mapping information will be updated so that the association has the current (inheriting) class as the source entity.
2138cc9383/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php (L384-L393)
This was added in 7dc8ef1db9 for [DDC-671](https://github.com/doctrine/orm/issues/5181).
The reason for this is that a mapped superclass is not an entity itself and has no table.
So, in the database, associations can only be from the inheriting entities' tables towards the referred-to target. This is also the reason for the limitation that only to-one associations may be added in mapped superclasses, since for those the database foreign key can be placed on the table(s) of the inheriting entities (and there may be more than one child class).
Neither the decision to update the `sourceEntity` nor the validation check should be based on `$parent->isMappedSuperclass`.
This only works in the simple case where the class hierarchy is `Mapped Superclass → Entity`.
The check is wrong when we have an inheritance hierarchy set up and the class hierarchy is `Base Entity → Mapped Superclass → Child Entity`.
Bug 1: The association should keep the root entity as the source. After all, in a JTI, the root table will contain the foreign key, and we need to base joins on that table when traversing `FROM LeafClass l JOIN l.target`.
Bug 2: Do not reject the to-many association declared in the base class. It is ok to have the reverse (owning) side point back to the base entity, as it would be if there were no mapped superclasses at all. The mapped superclass does not declare, add or otherwise interfere with the to-many association at all.
Base the decision to change the `sourceEntity` on `$mapping['inherited']` being set. This field points to the topmost _parent entity_ class in the ancestry tree where the relationship mapping appears for the first time.
When it is not set, the current class is the first _entity_ class in the hierarchy with that association. Since we are inheriting the relation, it must have been added in a mapped superclass above, but was not yet present in the nearest parent entity class.
In that case, it may only be a to-one association and the source entity needs to be updated.
(See #10396 for a clarification of the semantics of `inherited`.)
Here is a simplified example of the class hierarchy.
See the two tests added for more details – one is for checking the correct usage of a to-one association against/with the base class in JTI. The other is to test that a to-many association on the base class is not rejected.
I am sure that there are other tests that (still) cover the update of `sourceEntity` is happening.
```php
/**
* @Entity
*/
class AssociationTarget
{
/**
* @Column(type="integer")
* @Id
* @GeneratedValue
*/
public $id;
}
/**
* @Entity
* @InheritanceType("JOINED")
* @DiscriminatorColumn(name="discriminator", type="string")
* @DiscriminatorMap({"1" = "BaseClass", "2" = "LeafClass"})
*/
class BaseClass
{
/**
* @Column(type="integer")
* @Id
* @GeneratedValue
*/
public $id;
/**
* @ManyToOne(targetEntity="AssociationTarget")
*/
public $target;
}
/**
* @MappedSuperclass
*/
class MediumSuperclass extends BaseClass
{
}
/**
* @Entity
*/
class LeafClass extends MediumSuperclass
{
}
```
When querying `FROM LeafClass l`, it should be possible to `JOIN l.target`. This currently leads to an SQL error because the SQL join will be made via `LeafClass.target_id` instead of `BaseClass.target_id`. `LeafClass` is considered the `sourceEntity` for the association – which is wrong–, and so the foreign key field is expected to be in the `LeafClass` table (using JTI here).
Fixes#5998, fixes#7825.
I have removed the abstract entity class, since it is not relevant for the issue and took the discussion off course. Also, the discriminator map now contains all classes.
Added the second variant of the bug, namely that a to-many association would wrongly be rejected in the same situation.
The ArgumentTypeCoercion one is hiding issues we could address.
Addressing them will require changes that should go in the next minor,
so I'm moving them to the baseline instead. That way, we cannot
introduce more issues of this type in this file.
* 2.15.x:
Use more precise types for class strings (#10381)
PHPStan 1.9.8, Psalm 5.4.0 (#10382)
fix typo for missing a comma (#10377)
Docs: Removing `type: 'integer'` from mappings (#10368)
Docs: Moving *attributes* mapping to first position (#10364)
Docs: Deleting duplicate mapping example (#10363)
* 2.14.x:
PHPStan 1.9.8, Psalm 5.4.0 (#10382)
fix typo for missing a comma (#10377)
Docs: Removing `type: 'integer'` from mappings (#10368)
Docs: Moving *attributes* mapping to first position (#10364)
Docs: Deleting duplicate mapping example (#10363)
Yet another micro-PR ;-) - as requested at https://github.com/doctrine/orm/pull/10364#issuecomment-1370155521
I also changed `$currentPrice` from `float` to `int`, since IMO it's better to store prices as `int` (=cents).
Question: Is there a reason why most typehints are `int|null`, instead of `?int`? Should I change them?
These are the lowest hanging fruits I could find after running Rector: I
looked for files with a diff of 2 lines.
I did not include some changes that I find controversial, such as
marking some constants as final when we should maybe consider making
classes themselves final.
Following 1915dcd1e8, 0 is now used as a
default value, and Query::$firstResult is no longer nullable, but it
seems getFirstResult() was overlooked. The class is final, so this is no
breaking change.
* 2.15.x:
Shorter deprecation message (#10357)
Add Fully-Qualified class name in UnrecognizedField exception to ease debugging (#10342)
Include parameter types in hydration cache key generation (#10355)
Allow doctrine/instantiator 2 (#10351)
* 2.14.x:
Shorter deprecation message (#10357)
Add Fully-Qualified class name in UnrecognizedField exception to ease debugging (#10342)
Include parameter types in hydration cache key generation (#10355)
* 2.15.x:
Support of NOT expression from doctrine/collections ^2.1 (#10234)
Fix Psalm errors with Collection 2.1.2 (#10343)
Added warning about query cache in relation to parameters (#10276)
It is important to have the same version of all dependencies in dev and
in the CI, otherwise it makes it hard to have the right static analysis
baseline for every environment.
Since doctrine/annotations 1.10, autoloading is used when everything
else has failed. Using registerFile() has been deprecated in favor of
that later on.
Previously, only a predefined set of automatic mappings was allowed, such as int, float, boolean, DateTime etc.
With this extension, it is possible to supply custom TypedFieldMapper implementation which takes as parameter the ReflectionProperty of a given field and decides the appropriate mapping.
A new configuration option was added to set and get the TypedFieldMapper.
The old logic was moved into a class DefaultTypedFieldMapper which is used by default when no mapper is supplied.
The selected TypedFieldMapper is passed into ClassMetadataInfo constructor. If empty, the DefaultTypedFieldMapper is used.
There is also ChainTypedFieldMapper class which allows chaining multiple TypedFieldMappers and apply them in a cascade (always if a field gets type assigned by the earlier mapper in the list, it will not be changed later).
This commit adds enumType option to DiscriminatorColumn as well as support for custom DBAL types which use PHP enums for PHP values.
Previously, the enumType option was completely missing, but also even using custom types that used PHP enums would end up in exception because ObjectHydrator would try to convert enums to string using (string) explicit conversion.
Apart from hydrators, ClassMetadataBuilder was extended to support specifying enumType.
Documentation was updated.
Previously, array of enums were incorrectly compared in UoW::computeChangeSet() resulting always in false positive, since the original data for comparison is fetched using ReflectionEnumProperty::getValue(), which returns the enum values, not enum objects.
This fix ensures comparing the individual enum array members' values.
Enum fields as ID have worked for some time, but referencing these fields from other entities in association mappings such as OneToOne, ManyToOne and ManyToMany resulted in fatal error as there was an attempt to convert enum value to string improperly.
Previously different hydrators would store the original data for enum
fields in different ways, the SimpleObjectHydrator would keep them as
strings while other hydrators would convert then to native php enums.
This would make the data in the internal UnitOfWork::$originalEntityData
array inconsistent which could've caused problems in the long run.
Now, all hydrators convert enum fields to native php enums ensuring the
original data is always consistent regardless of the hydrator used.
* Migrate AST namespace to PHP 8 syntax
* Use typed properties when type is non-object
We know the phpdoc types in that namespace are pretty messed up, but it
should be safe to assume that's only the case when the type is an object
type.
In 68bc00b6c6, while fixing coding style,
I introduced many, many use statements in that file. Using
half-qualified names sometimes, and unqualified names some other times
makes no sense.
If AST\ is used at least once, use it always.
It is not stable yet, but should be good enough, and this will help
taking care of #10118
Let us baseline all the new issues, just because they are new does not
mean they are more important than already-baselined errors. Besides, it
is more important to have higher standards for new code than to get an
increased baseline.
* 2.14.x:
Remove fragile assertions (#10239)
update help for RunDqlCommand (#10233)
Make the code easier to statically analyse
Widen parameter type
Document property as non-nullable
This exception is used in two places, one where $generatedMode is
clearly a string, and another where typing it as a string will cause a
type error, because $generatedMode is supposed to be an int there.
Inside the Query\AST namespace, many classes use public properties that
are supposed to be set from outside the class. Let us ignore
PropertyNotSetInConstructor for that entire namespace instead of doing
it on a case by case basis.
* 2.14.x:
Add a constructor to CacheKey (#10212)
Psalm 4.30.0, PHPStan 1.9.2 (#10213)
Allow "Expr\Func" as condition in join (#10202)
refactor: use list type in SchemaTool (#10199)
* Remove file used by annotation registry
* Remove meta-annotations
They should have been removed when the annotation driver was removed.
* Remove unneeded coding standard rule exclusion
* Remove annotation documentation of indexBy
* 2.14.x:
Deprecate the Annotation interface (#10178)
Bump CI to PHP 8.2 and latest database versions (#10180)
Remove reference to deprecated DriverChain from docs (#10179)
* 2.14.x:
PHPStan 1.8.11 (#10182)
Add isMemberOf and isInstanceOf to Expr helper list (#10104)
Migrate more references to annotations (#10176)
Fix grammer in working-with-objects (#10120)
Automap events in AttachEntityListenersListener (#10122)
Adapt use statements to the code (#10174)
I used the following config:
<?php
declare(strict_types=1);
use Rector\Config\RectorConfig;
use Rector\Doctrine\Set\DoctrineSetList;
return function (RectorConfig $rectorConfig): void {
$rectorConfig->paths([
__DIR__ . '/tests',
]);
$rectorConfig->sets([
DoctrineSetList::ANNOTATIONS_TO_ATTRIBUTES,
]);
};
* 2.14.x:
Deprecate EntityManager::create() (#9961)
Address deprecation of SchemaDiff::toSaveSql()
Address deprecation of SchemaDiff::toSql()
Use error style for notifications
Fix calls to AbstractSchemaManager::createSchema() (#10165)
Fix build with DBAL 3.5 (#10163)
Adjust comments (#10160)
Deprecate methods related to the annotation driver
Use correct link
Deprecate annotations
Remove trailing whitespace
Migrate more documentation towards attributes
Remove wrong sentence
Do not export phpstan stuff (#10154)
stderr is not a great name for something that is not meant to be
processed (piped into) a program, but to be read by humans.
Most commands should use stderr, and some of them should partially use
stdout, typically when dumping SQL requests.
The previous rework missed some details / left the documentation in an
inconsistent state. Searching for "annotation" case insensitively allows
to find discrepancies and forgotten things.
* 2.14.x:
Modernize documentation code
Add CI jobs for SQLite3 driver (#10141)
Fix type doc blocks in annotation classes (#10145)
Fix FieldMapping for generated key (#10144)
Stop triggering static analysis workflows on tests
- migrate to attributes;
- add helpful phpdoc annotations;
- use typed properties;
- add type declarations.
Co-authored-by: Alexander M. Turek <me@derrabus.de>
The downside of this is that we will have to tweak the settings so that
no job is required anymore.
The upside is that builds should be faster, and less resource-intensive.
__sleep() does not list these properties as suposed to be serialized.
This means we have to assert that it is not in the many scenarios where
we resort to these properties. Introducing a private method allows us to
centralize the assertions.
* Add missing variable name in param phpdoc
* Use psalm-assert instead of psalm-param
The parameter can in fact be any string, but if an exception is not
thrown, then it was a valid value.
* 2.14.x:
PHPStan 1.8.5, Psalm 4.27.0 (#10033)
Fix EnumType not being hydrated with HYDRATE_ARRAY (#9995)
"Strange" return lines in documentation of inheritance-mapping.rst (#10027)
More strange break lines in inheritance-mapping.rst (#10028)
Add phpdoc for discriminatorColumn
* 2.14.x:
Bump Ubuntu version and shared workflows (#10020)
Make paginator covariant (#10002)
Fail gracefully if EM could not be constructed in tests (#10008)
* 2.14.x:
Bump coding standard to 9.0.2
Fix tests for doctrine/common 3.4
Fix static analysis errors for Collections 1.7
Fix type in docs (#9994)
Improve orphan removal documentation - recommend using cascade=persist (#9848)
* 2.14.x:
Backport type fixes for EntityListenerResolver (#9977)
Undeprecate LifecycleEventArgs (#9980)
Update documentation to not use deprecated method (#9979)
Deprecating this class means that users using SA tools have to
update their code specifying the ObjectManager implementation.
Instead of this, dedicated classes for each event should be
created before deprecating this class.
* 2.13.x:
Address DBAL 3.4 deprecations (#9969)
Improve phpdoc for ClassMetadataInfo (#9965)
Fix build (#9964)
fix: class normalisation test (#9966)
Support native enum hydration when using `NEW` operator (#9936)
* 2.13.x:
Deprecate QueryBuilder APIs exposing its internal state (#9945)
Update branch info in README and .doctrine-project.json (#9943)
Psalm 4.25.0, PHPStan 1.8.2 (#9941)
Stop passing event manager to constructor (#9938)
Use a more precise phpdoc for ClassMetadataInfo::versionField than mixed (#9937)
Make EntityManager `@final` and its constructor public
* 2.13.x:
Add helper function TreeWalkerAdapter::getMetadataForDqlAlias() (#9932)
Simplify LanguageRecognitionTest (#9930)
test/added test for foreign keys with custom id object types
Widen types for DiscriminatorMap (#9922)
schema tool: remove useless text from --dump-sql output
Update IdentifierFlattener.php
Update IdentifierFlattener.php
* 2.13.x:
Deprecate omitting the alias in QueryBuilder (#9765)
Run tests on PHP 8.2 (#9840)
PHPStan 1.7.13 (#9844)
Flip conditional extension of legacy AnnotationDriver class (#9843)
PHP CodeSniffer 3.7 (#9842)
Make Reflection available to ConvertMappingCommand (#9619)
Add missing property declaration
Use proper API for introspection of tables
This causes "Cannot assign string to property
Doctrine\Tests\Models\Cache\ComplexAction::$action2 of type
Doctrine\Tests\Models\Cache\Action" for some reason.
This causes the following error for some reason:
Typed property Doctrine\Tests\ORM\Functional\TrainOrder::$train must not
be accessed before initialization
* 2.13.x:
Add missing import (#9796)
Deprecate calling setters without arguments (#9791)
Move duplicate fixture into dedicated file (#9789)
MockTreeWalker should be an SqlWalker (#9790)
* 2.13.x:
Make phpdoc more precise
Deprecate setting fetch mode to random integers
Prepare split of output walkers and tree walkers (#9786)
PHPStan 1.7.0 (#9785)
Deprecate passing null to Query::setDQL()
Kill call_user_func(_array) (#9780)
Fix wrong types for AbstractQuery and child classes (#9774)
Document callable as possible
Remove override phpdoc tag
Add use statement (#9769)
This allows to get rid of tearDown(), which contained a special handling
that is no longer necessary since we switched away from explicitely
managed sequences, and caused the test suite to fail.
* Remove disconnected class metadata factory
It is unused apart from a tests where it is easily replaced.
* Remove ClassMetadataInfo
It has been deprecated for a long, long time.
* 2.12.x:
Indicate support for doctrine/persistence 3 (#9656)
Fix tests for enum ID hydration (#9658)
Revert "Use charset/collation from column or table default when creating relations (#9636)"
Fix test file/class names (#9649)
* 2.12.x:
Update psalm.xml
PHPStan 1.5.0 (#9607)
Remove Sphinx config
Indicate what feature is deprecated
Implement int-mask-of where appropriate
Use correct syntax for external links
Update XmlExporter.php - Type problem in php8.x (#9589)
Ignore deprecation from Persistence
Stands with Ukraine (#9567)
Use internal links when self-referencing
Link to docs for the stable version
* 2.12.x:
Leverage MemcachedAdapter::isSupported() (#9578)
Baseline Psalm errors caused by DBAL 3.3.3 (#9577)
Make sure MemcachedAdapter is supported before tring to use it (#9574)
Fixing `:doc:` link (#9569)
Adding PHP attributes (#9555)
Remove reference to removed class
* 2.12.x:
Make creating test models more straightforward
Trigger the desired code path
Fix syntax typo in attributes reference (#9513)
Constructor-Argument "options" has the same type as the associated property. (#9501)
* 2.12.x:
Deprecate methods removed in 3.0 (#9475)
Skip tests related to PersistentObject if that class is missing (#9472)
Run Postgres 14 and MariaDB 10.6 in CI (#9470)
* 2.12.x:
Check requirements for metadata drivers (#9459)
PDO is not a required extension (#9457)
Check requirements for metadata drivers (#9452)
Remove trailing underscore (#9446)
* 2.12.x:
Introduce DoctrineSetup as a replacement for Setup (#9443)
Introduce __unserialize behaviour in docs (#9390)
Adapt test logic to PHP and SQLite II (#9442)
Use the identify generator strategy
Added php 8.1 to CI
Psalm 4.19.0, PHPStan 1.4.3 (#9438)
Ignore PHPUnit result cache everywhere (#9425)
* 2.12.x:
Add support for PHP 8.1 enums in embedded classes (#9419)
Switch tests to the middleware logging system (#9418)
Added class-string typehint on $targetEntity (#9415)
Allow DiscriminatorColumn with length=0 (#9410)
Move UnderscoreNamingStrategyTest to correct namespace (#9414)
* 2.12.x:
Deprecate MultiGetRegion (#9397)
Fix type on loadCacheEntry (#9398)
Update baselines for DBAL 3.3 (#9393)
Accessing private properties and methods from the same class is forbidden (#9311)
Expose enumType to DBAL to make native DB Enum possible (#9382)
* 2.12.x:
Allow using Enum from different namespace than Entity (#9384)
Corrected ORM version and added missing dependency (#9386)
PHPStan 1.4.0 (#9385)
[GH-9380] Bugfix: Delegate ReflectionEnumProperty::getAttributes(). (#9381)
Support enum cases as parameters (#9373)
Add detach as of list cascade-all operations (#9357)
* 2.11.x:
Use EntityManagerInterface in type declarations (#9325)
Add errors caused by the lexer update to the baselines (#9360)
Generated/Virtual Columns: Insertable / Updateable (#9118)
Remove the composer/package-versions-deprecated package
Relax assertion to include null as possible outcome (#9355)
* 2.11.x:
Leverage generic ObjectManagerDecorator (#9312)
Fix WhereInWalker description to better describe the behaviour of this class (#9268)
Regenerate Psalm baseline
Update Psalm baseline for Persistence 2.3 (#9349)
Support readonly properties for read operations (#9316)
* 2.11.x:
PHPStan 1.3.3, Psalm 4.18.1
Remove Psalm job for analyzing DBAL 2
Use the readonly annotation (#9340)
Add support for custom types with requireSQLConversion and ResultSetMappingBuilder::generateSelectClause()
PSR-6 second level cache
Fix type errors in AbstractQuery and QueryBuilder (#9275)
Mark columnName as always set
Add support for PHP 8.1 enums.
Remove ignore rules for issues fixed upstream (#9336)
[GH-9277] deprecate php driver (#9309)
* 2.11.x:
Added runtime deprecation to `UnitOfWork::commit()` and `clear()` (#9327)
Document return type of getEntityState() (#9328)
Fix broken type declaration (#9330)
Enable some previously disabled PHPCS rules (#9324)
* 2.11.x:
Run static analysis with language level PHP 8.1 (#9314)
Document LockMode enums (#9319)
Document PHPUnit mocks with intersection types (#9318)
Run PHP CodeSniffer on PHP 8.1 (#9317)
Psalm 4.17.0 (#9315)
Run static analysis on PHP 8.1 (#9310)
* 2.11.x:
Leverage get_debug_type() (#9297)
Fix return type (#9295)
Synchronize Psalm baseline (#9296)
Fix union type on QueryExpressionVisitorTest::testWalkComparison() (#9294)
Allow arithmetic expressions within IN operator (#9242)
Bump reusable workflows
* 2.11.x:
Enable UnusedUse sniff again (#9267)
Whitelist composer plugins used by this repository (#9286)
Fix XML export for `change-tracking-policy` (#9285)
Allow symfony/cache 6 (#9283)
Put actual value instead of index inside $originalEntityData. (#9244)
Fix return types of cache interfaces (#9271)
Better explain limitations of DQL "DELETE" (#9281)
Fix docblocks on nullable EM properties (#9273)
Docs: use canonical order for phpdoc tags, add missed semicolon (#9190)
Make PrimaryReadReplicaConnection enforcement explicit (#9239)
Regenerate Psalm baseline (#9272)
Improve compatibility with Doctrine DBAL 4 (#9266)
[docs] Fix wording for attributes=>parameters. (#9265)
Support for nesting attributes with PHP 8.1 (#9241)
Revert "Fix SchemaValidator with abstract child class in discriminator map (#9096)" (#9262)
Docs: consistency for FQCN, spacing, etc (#9232)
* 2.11.x:
Add missing deprecations for YAML metadata mapping (#9206)
Drop support for DBAL 3.1
Psalm 4.13.1, PHPStan 1.2.0 (#9204)
Add a psalm type for field mapping
Use `equal to` instead of `equal of` in `assertSqlGeneration()` (#9195)
Adding Attributes code block (#9161)
Currently to get count of all items there is need to provide empty array
to count() method as $criteria parameter is required. I believe there
shouldn't be a need to provide it if I want to count all Entities
without any criteria.
about: Have you encountered an issue during upgrade? 💣
---
<!--
Before reporting a BC break, please consult the upgrading document to make sure it's not an expected change: https://github.com/doctrine/orm/blob/2.9.x/UPGRADE.md
-->
### BC Break Report
<!-- Fill in the relevant information below to help triage your issue. -->
| Q | A
|------------ | ------
| BC Break | yes
| Version | x.y.z
#### Summary
<!-- Provide a summary describing the problem you are experiencing. -->
#### Previous behavior
<!-- What was the previous (working) behavior? -->
#### Current behavior
<!-- What is the current (broken) behavior? -->
#### How to reproduce
<!--
Provide steps to reproduce the BC break.
If possible, also add a code snippet with relevant configuration, entity mappings, DQL etc.
Adding a failing Unit or Functional Test would help us a lot - you can submit it in a Pull Request separately, referencing this bug report.
but be aware that it is deprecated since PHP 8.1. We do not recommend its usage.
- PHP 7.4 introduces :doc:`the new magic method <https://php.net/manual/en/language.oop5.magic.php#object.unserialize>`
``__unserialize``, which changes the execution priority between
``__wakeup`` and itself when used. This can cause unexpected behaviour in
an Entity.
- An entity class must not be final nor read-only but
it may contain final methods or read-only properties.
- Any two entity classes in a class hierarchy that inherit
directly or indirectly from one another must not have a mapped
property with the same name. That is, if B inherits from A then B
must not have a mapped field with the same name as an already
mapped field that is inherited from A.
- An entity cannot make use of func_get_args() to implement variable parameters.
Generated proxies do not support this for performance reasons and your code might
actually fail to work when violating this restriction.
- Entity cannot access private/protected properties/methods of another entity of the same class or :doc:`do so safely <../cookbook/accessing-private-properties-of-the-same-class-from-different-instance>`.
Entities support inheritance, polymorphic associations, and
polymorphic queries. Both abstract and concrete classes can be
@@ -113,6 +99,25 @@ classes, and non-entity classes may extend entity classes.
never calls entity constructors, thus you are free to use them as
you wish and even have it require arguments of any type.
Mapped Superclasses
~~~~~~~~~~~~~~~~~~~
A mapped superclass is an abstract or concrete class that provides
persistent entity state and mapping information for its subclasses,
but which is not itself an entity.
Mapped superclasses are explained in greater detail in the chapter
on :doc:`inheritance mapping </reference/inheritance-mapping>`.
Transient Classes
~~~~~~~~~~~~~~~~~
The term "transient class" appears in some places in the mapping
drivers as well as the code dealing with metadata handling.
A transient class is a class that is neither an entity nor a mapped
superclass. From the ORM's point of view, these classes can be
completely ignored, and no class metadata is loaded for them at all.
Entity states
~~~~~~~~~~~~~
@@ -159,17 +164,13 @@ Serializing entities
Serializing entities can be problematic and is not really
recommended, at least not as long as an entity instance still holds
references to proxy objects or is still managed by an
EntityManager. If you intend to serialize (and unserialize) entity
instances that still hold references to proxy objects you may run
into problems with private properties because of technical
limitations. Proxy objects implement ``__sleep`` and it is not
possible for ``__sleep`` to return names of private properties in
parent classes. On the other hand it is not a solution for proxy
objects to implement ``Serializable`` because Serializable does not
work well with any potential cyclic object references (at least we
did not find a way yet, if you did, please contact us). The
``Serializable`` interface is also deprecated beginning with PHP 8.1.
references to proxy objects or is still managed by an EntityManager.
By default, serializing proxy objects does not initialize them. On
unserialization, resulting objects are detached from the entity
manager and cannot be initialiazed anymore. You can implement the
``__serialize()`` method if you want to change that behavior, but
then you need to ensure that you won't generate large serialized
object graphs and take care of circular associations.
The EntityManager
~~~~~~~~~~~~~~~~~
@@ -203,5 +204,3 @@ typical implementation of the
to keep track of all the things that need to be done the next time
``flush`` is invoked. You usually do not directly interact with a
``UnitOfWork`` but with the ``EntityManager`` instead.
@@ -180,10 +180,10 @@ not need to lazy load the association with another query.
Doctrine allows you to walk all the associations between
all the objects in your domain model. Objects that were not already
loaded from the database are replaced with lazyload proxy
instances. Non-loaded Collections are also replaced by lazy-load
loaded from the database are replaced with lazy-loading proxy
instances. Non-loaded Collections are also replaced by lazy-loading
instances that fetch all the contained objects upon first access.
However relying on the lazy-load mechanism leads to many small
However relying on the lazy-loading mechanism leads to many small
queries executed against the database, which can significantly
affect the performance of your application. **Fetch Joins** are the
solution to hydrate most or all of the entities that you need in a
@@ -319,11 +319,11 @@ With Nested Conditions in WHERE Clause:
<?php
$query=$em->createQuery('SELECT u FROM ForumUser u WHERE (u.username = :name OR u.username = :name2) AND u.id = :id');
$query->setParameters(array(
$query->setParameters([
'name'=>'Bob',
'name2'=>'Alice',
'id'=>321,
));
]);
$users=$query->getResult();// array of ForumUser objects
With COUNT DISTINCT:
@@ -464,6 +464,11 @@ hierarchies:
$query=$em->createQuery('SELECT u FROM Doctrine\Tests\Models\Company\CompanyPerson u WHERE u INSTANCE OF Doctrine\Tests\Models\Company\CompanyEmployee');
$query=$em->createQuery('SELECT u FROM Doctrine\Tests\Models\Company\CompanyPerson u WHERE u INSTANCE OF ?1');
$query=$em->createQuery('SELECT u FROM Doctrine\Tests\Models\Company\CompanyPerson u WHERE u NOT INSTANCE OF ?1');
Get all users visible on a given website that have chosen certain gender:
@@ -519,8 +524,8 @@ when the DQL is switched to an arbitrary join.
aggregation (GROUP BY)
Partial Object Syntax
^^^^^^^^^^^^^^^^^^^^^
Partial Hydration Syntax
^^^^^^^^^^^^^^^^^^^^^^^^
By default when you run a DQL query in Doctrine and select only a
subset of the fields for a given entity, you do not receive objects
@@ -528,7 +533,7 @@ back. Instead, you receive only arrays as a flat rectangular result
set, similar to how you would if you were just using SQL directly
and joining some data.
If you want to select partial objects you can use the ``partial``
If you want to select partial objects or fields in array hydration you can use the ``partial``
DQL keyword:
..code-block::php
@@ -537,12 +542,13 @@ DQL keyword:
$query=$em->createQuery('SELECT partial u.{id, username} FROM CmsUser u');
$users=$query->getResult();// array of partially loaded CmsUser objects
You use the partial syntax when joining as well:
You can use the partial syntax when joining as well:
..code-block::php
<?php
$query=$em->createQuery('SELECT partial u.{id, username}, partial a.{id, name} FROM CmsUser u JOIN u.articles a');
$usersArray=$query->getArrayResult();// array of partially loaded CmsUser and CmsArticle fields
$users=$query->getResult();// array of partially loaded CmsUser objects
"NEW" Operator Syntax
@@ -582,7 +588,112 @@ And then use the ``NEW`` DQL keyword :
$query=$em->createQuery('SELECT NEW CustomerDTO(c.name, e.email, a.city, SUM(o.value)) FROM Customer c JOIN c.email e JOIN c.address a JOIN c.orders o GROUP BY c');
$users=$query->getResult();// array of CustomerDTO
Note that you can only pass scalar expressions to the constructor.
$query=$em->createQuery('SELECT NEW CustomerDTO(c.name, e.email, NEW AddressDTO(a.street, a.city, a.zip)) FROM Customer c JOIN c.email e JOIN c.address a');
$users=$query->getResult();// array of CustomerDTO
Note that you can only pass scalar expressions or other Data Transfer Objects to the constructor.
If you use your data transfer objects for multiple queries, and you would rather not have to
specify arguments that precede the ones you are really interested in, you can use named arguments.
Consider the following DTO, which uses optional arguments:
..code-block::php
<?php
classCustomerDTO
{
publicfunction__construct(
publicstring|null$name=null,
publicstring|null$email=null,
publicstring|null$city=null,
publicmixed|null$value=null,
publicAddressDTO|null$address=null,
){
}
}
You can specify arbitrary arguments in an arbitrary order by using the named argument syntax, and the ORM will try to match argument names with the selected column names.
The syntax relies on the NAMED keyword, like so:
..code-block::php
<?php
$query=$em->createQuery('SELECT NEW NAMED CustomerDTO(a.city, c.name) FROM Customer c JOIN c.address a');
$users=$query->getResult();// array of CustomerDTO
To define a custom name for a DTO constructor argument, you can either alias the column with the ``AS`` keyword.
The ``NAMED`` keyword must precede all DTO you want to instantiate :
..code-block::php
<?php
$query=$em->createQuery('SELECT NEW NAMED CustomerDTO(c.name, NEW NAMED AddressDTO(a.street, a.city, a.zip) AS address) FROM Customer c JOIN c.address a');
$users=$query->getResult();// array of CustomerDTO
If two arguments have the same name, a ``DuplicateFieldException`` is thrown.
If a field cannot be matched with a property name, a ``NoMatchingPropertyException`` is thrown. This typically happens when using functions without aliasing them.
You can hydrate an entity nested in a DTO :
..code-block::php
<?php
$query=$em->createQuery('SELECT NEW CustomerDTO(c.name, a AS address) FROM Customer c JOIN c.address a');
$users=$query->getResult();// array of CustomerDTO
@@ -28,10 +28,11 @@ table alias of the SQL table of the entity.
In the case of joined or single table inheritance, you always get passed the ClassMetadata of the
inheritance root. This is necessary to avoid edge cases that would break the SQL when applying the filters.
Parameters for the query should be set on the filter object by
``SQLFilter#setParameter()``. Only parameters set via this function can be used
in filters. The ``SQLFilter#getParameter()`` function takes care of the
proper quoting of parameters.
For the filter to correctly function, the following rules must be followed. Failure to do so will lead to unexpected results from the query cache.
1. Parameters for the query should be set on the filter object by ``SQLFilter#setParameter()`` before the filter is used by the ORM ( i.e. do not set parameters inside ``SQLFilter#addFilterConstraint()`` function ).
2. The filter must be deterministic. Don't change the values base on external inputs.
The ``SQLFilter#getParameter()`` function takes care of the proper quoting of parameters.
We try to make using Doctrine2 a very pleasant experience.
We try to make using Doctrine ORM a very pleasant experience.
Therefore we think it is very important to be honest about the
current limitations to our users. Much like every other piece of
software Doctrine2 is not perfect and far from feature complete.
software the ORM is not perfect and far from feature complete.
This section should give you an overview of current limitations of
Doctrine ORM as well as critical known issues that you should know
about.
@@ -65,15 +65,6 @@ Where the ``attribute_name`` column contains the key and
The feature request for persistence of primitive value arrays
`is described in the DDC-298 ticket <https://github.com/doctrine/orm/issues/3743>`_.
Cascade Merge with Bi-directional Associations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There are two bugs now that concern the use of cascade merge in combination with bi-directional associations.
Make sure to study the behavior of cascade merge if you are using it:
-`DDC-875 <https://github.com/doctrine/orm/issues/5398>`_ Merge can sometimes add the same entity twice into a collection
-`DDC-763 <https://github.com/doctrine/orm/issues/5277>`_ Cascade merge on associated entities can insert too many rows through "Persistence by Reachability"
Custom Persisters
~~~~~~~~~~~~~~~~~
@@ -130,10 +121,63 @@ included in the core of Doctrine ORM. However there are already two
extensions out there that offer support for Nested Set with
* query = "SELECT u.id AS u_id, u.name AS u_name, u.status AS u_status, a.id AS a_id, a.zip AS a_zip, a.country AS a_country, COUNT(p.phonenumber) AS numphones FROM users u INNER JOIN addresses a ON u.id = a.user_id INNER JOIN phonenumbers p ON u.id = p.user_id GROUP BY u.id, u.name, u.status, u.username, a.id, a.zip, a.country ORDER BY u.username"
<query>SELECT u.id AS u_id, u.name AS u_name, u.status AS u_status, a.id AS a_id, a.zip AS a_zip, a.country AS a_country, COUNT(p.phonenumber) AS numphones FROM users u INNER JOIN addresses a ON u.id = a.user_id INNER JOIN phonenumbers p ON u.id = p.user_id GROUP BY u.id, u.name, u.status, u.username, a.id, a.zip, a.country ORDER BY u.username</query>
query:SELECT u.id AS u_id, u.name AS u_name, u.status AS u_status, a.id AS a_id, a.zip AS a_zip, a.country AS a_country, COUNT(p.phonenumber) AS numphones FROM users u INNER JOIN addresses a ON u.id = a.user_id INNER JOIN phonenumbers p ON u.id = p.user_id GROUP BY u.id, u.name, u.status, u.username, a.id, a.zip, a.country ORDER BY u.username
sqlResultSetMappings:
mappingMultipleJoinsEntityResults:
name:mappingMultipleJoinsEntityResults
columnResult:
0:
name:numphones
entityResult:
0:
entityClass:__CLASS__
fieldResult:
0:
name:id
column:u_id
1:
name:name
column:u_name
2:
name:status
column:u_status
1:
entityClass:Address
fieldResult:
0:
name:id
column:a_id
1:
name:zip
column:a_zip
2:
name:country
column:a_country
Things to note:
- The resultset mapping declares the entities retrieved by this native query.
- Each field of the entity is bound to a SQL alias (or column name).
- All fields of the entity including the ones of subclasses
and the foreign key columns of related entities have to be present in the SQL query.
- Field definitions are optional provided that they map to the same
column name as the one declared on the class property.
- ``__CLASS__`` is an alias for the mapped class
In the above example,
the ``fetchJoinedAddress`` named query use the joinMapping result set mapping.
This mapping returns 2 entities, User and Address, each property is declared and associated to a column name,
actually the column name retrieved by the query.
Let's now see an implicit declaration of the property / column.
In this example, we only describe the entity member of the result set mapping.
The property / column mappings is done using the entity mapping values.
In this case the model property is bound to the model_txt column.
If the association to a related entity involve a composite primary key,
a @FieldResult element should be used for each foreign key column.
The @FieldResult name is composed of the property name for the relationship,
followed by a dot ("."), followed by the name or the field or property of the primary key.
..configuration-block::
..code-block::php
<?php
namespaceMyProject\Model;
/**
* @NamedNativeQueries({
* @NamedNativeQuery(
* name = "fetchJoinedAddress",
* resultSetMapping= "mappingJoinedAddress",
* query = "SELECT u.id, u.name, u.status, a.id AS a_id, a.country AS a_country, a.zip AS a_zip, a.city AS a_city FROM users u INNER JOIN addresses a ON u.id = a.user_id WHERE u.username = ?"
<query>SELECT u.id, u.name, u.status, a.id AS a_id, a.country AS a_country, a.zip AS a_zip, a.city AS a_city FROM users u INNER JOIN addresses a ON u.id = a.user_id WHERE u.username = ?</query>
query:SELECT u.id, u.name, u.status, a.id AS a_id, a.country AS a_country, a.zip AS a_zip, a.city AS a_city FROM users u INNER JOIN addresses a ON u.id = a.user_id WHERE u.username = ?
sqlResultSetMappings:
mappingJoinedAddress:
entityResult:
0:
entityClass:__CLASS__
fieldResult:
0:
name:id
1:
name:name
2:
name:status
3:
name:address.id
column:a_id
4:
name:address.zip
column:a_zip
5:
name:address.city
column:a_city
6:
name:address.country
column:a_country
If you retrieve a single entity and if you use the default mapping,
you can use the resultClass attribute instead of resultSetMapping:
@@ -612,10 +622,10 @@ Now two examples of what happens when you remove the references:
$em->flush();
In this case you have not only changed the ``Contact`` entity itself but
you have also removed the references for standing data and as well as one
address reference. When flush is called not only are the references removed
but both the old standing data and the one address entity are also deleted
In this case you have not only changed the ``Contact`` entity itself but
you have also removed the references for standing data and as well as one
address reference. When flush is called not only are the references removed
but both the old standing data and the one address entity are also deleted
from the database.
.._filtering-collections:
@@ -708,6 +718,7 @@ methods:
*``andX($arg1, $arg2, ...)``
*``orX($arg1, $arg2, ...)``
*``not($expression)``
*``eq($field, $value)``
*``gt($field, $value)``
*``lt($field, $value)``
@@ -725,6 +736,35 @@ methods:
..note::
There is a limitation on the compatibility of Criteria comparisons.
You have to use scalar values only as the value in a comparison or
the behaviour between different backends is not the same.
Depending on whether the collection has already been loaded from the
database or not, criteria matching may happen at the database/SQL level
or on objects in memory. This may lead to different results and come
surprising, for example when a code change in one place leads to a collection
becoming initialized and, as a side effect, returning a different result
or even breaking a ``matching()`` call somewhere else. Also, collection
initialization state in practical use cases may differ from the one covered
in unit tests.
Database level comparisons are based on scalar representations of the values
stored in entity properties. The field names passed to expressions correspond
to property names. Comparison and sorting may be affected by
database-specific behavior. For example, MySQL enum types sort by index position,
not lexicographically by value.
In-memory handling is based on the ``Selectable`` API of `Doctrine Collections <https://www.doctrine-project.org/projects/doctrine-collections/en/stable/index.html#matching>`.
In this case, field names passed to expressions are being used to derive accessor
method names. Strict type comparisons are used for equal and not-equal checks,
and generally PHP language rules are being used for other comparison operators
or sorting.
As a general guidance, for consistent results use the Criteria API with scalar
values only. Note that ``DateTime`` and ``DateTimeImmutable`` are two predominant
examples of value objects that are *not* scalars.
Refrain from using special database-level column types or custom Doctrine Types
that may lead to database-specific comparison or sorting rules being applied, or
to database-level values being different from object field values.
Provide accessor methods for all entity fields used in criteria expressions,
and implement those methods in a way that their return value is the
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
