Problems with Bulk Updates using Iterator #5492

New Issue

admin · 2026-01-22T15:09:05+01:00

admin commented

2026-01-22 15:09:05 +01:00

Originally created by @ncirocco on GitHub (Apr 3, 2017).

Originally assigned to: @Ocramius on GitHub.

$entities = $this->repo->findIterable();

$i = 0;
while (false !== ($row = $entities->next())) {
    $entity = $row[0];
    $entity->setStatus(1);

    if (($i % 5) === 0) {
        $this->em->flush();
        $this->em->clear();
    }

    $i++;
}

$this->em->flush();
$this->em->clear();

Using the previous code the only entities that are being successfully updated are the ones that the if condition is true during their iteration. The ones that were modified but flush was not called during their iteration are not being updated.
Also the remaining ones that should be flushed outside of the while are not being flushed successfully.

based on: http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/batch-processing.html#iterating-results

For example:
given the entities with id range 1-12

1 is updated
2 is not updated
3 is not updated
4 is not updated
5 is updated 
6 is not updated
7 is not updated
8 is not updated
9 is not updated
10 is updated
11 is not updated
12 is not updated

PHP version: 7.0.17

Doctrine
doctrine/annotations v1.2.7
doctrine/cache v1.4.2
doctrine/collections v1.3.0
doctrine/common v2.5.1
doctrine/data-fixtures v1.1.1
doctrine/dbal v2.4.4
doctrine/doctrine-bundle v1.5.2
doctrine/doctrine-cache-bundle v1.0.1
doctrine/doctrine-fixtures-bundle v2.2.1
doctrine/doctrine-migrations-bundle 1.0.1
doctrine/inflector v1.0.1
doctrine/instantiator 1.0.5
doctrine/lexer v1.0.1
doctrine/migrations dev-master 087f738
doctrine/orm v2.4.8

Originally created by @ncirocco on GitHub (Apr 3, 2017). Originally assigned to: @Ocramius on GitHub. ```php $entities = $this->repo->findIterable(); $i = 0; while (false !== ($row = $entities->next())) { $entity = $row[0]; $entity->setStatus(1); if (($i % 5) === 0) { $this->em->flush(); $this->em->clear(); } $i++; } $this->em->flush(); $this->em->clear(); ``` Using the previous code the only entities that are being successfully updated are the ones that the `if` condition is `true` during their iteration. The ones that were modified but `flush` was not called during their iteration are not being updated. Also the remaining ones that should be flushed outside of the while are not being flushed successfully. based on: http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/batch-processing.html#iterating-results For example: given the entities with id range 1-12 ``` 1 is updated 2 is not updated 3 is not updated 4 is not updated 5 is updated 6 is not updated 7 is not updated 8 is not updated 9 is not updated 10 is updated 11 is not updated 12 is not updated ``` **PHP version**: 7.0.17 **Doctrine** doctrine/annotations v1.2.7 doctrine/cache v1.4.2 doctrine/collections v1.3.0 doctrine/common v2.5.1 doctrine/data-fixtures v1.1.1 doctrine/dbal v2.4.4 doctrine/doctrine-bundle v1.5.2 doctrine/doctrine-cache-bundle v1.0.1 doctrine/doctrine-fixtures-bundle v2.2.1 doctrine/doctrine-migrations-bundle 1.0.1 doctrine/inflector v1.0.1 doctrine/instantiator 1.0.5 doctrine/lexer v1.0.1 doctrine/migrations dev-master 087f738 doctrine/orm v2.4.8

admin added the Bug Invalid Incomplete labels 2026-01-22 15:09:05 +01:00

admin closed this issue

2026-01-22 15:09:05 +01:00

admin commented

2026-01-22 15:09:06 +01:00

@Ocramius commented on GitHub (Apr 3, 2017):

this should be reproduced with a test case
your $this->em->clear(); causes the entities to be cleared, so there's no way to know whether some of those entities were cached inside the hydrator due to fetch-join operations, or whether they were completely detached (that's what you should assert upon)
use a tested utility that does this for you, and provides you with fresh entities at each iteration, like https://github.com/Ocramius/DoctrineBatchUtils/

Closing as invalid

@Ocramius commented on GitHub (Apr 3, 2017): 1. this should be reproduced with a test case 2. your `$this->em->clear();` causes the entities to be cleared, so there's no way to know whether some of those entities were cached inside the hydrator due to fetch-join operations, or whether they were completely detached (that's what you should assert upon) 3. use a tested utility that does this for you, and provides you with fresh entities at each iteration, like https://github.com/Ocramius/DoctrineBatchUtils/ Closing as `invalid`

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: doctrine/archived-orm#5492