doctrine/archived-orm
1
0
Fork 0
mirror of https://github.com/doctrine/orm.git synced 2026-03-23 22:42:18 +01:00
Code Issues 5.3k Packages Projects Releases 100 Wiki Activity

[Question] Why is memory still increasing by using EM flush ? #5861

New Issue
Closed
opened 2026-01-22 15:20:20 +01:00 by admin · 15 comments
admin commented 2026-01-22 15:20:20 +01:00
Owner
Copy Link

Originally created by @poolerMF on GitHub (Jan 25, 2018).

Originally assigned to: @Ocramius on GitHub.

example:

echo "first :".memory_get_usage(true)."<br>";

$em = $this->getDoctrine();

for($y = 0;$y<5;$y++) {
	$user = $em->getRepository(User::class)->find(1);

	for($x = 0;$x<500+$y;$x++) {
		$token = new CentralToken();
		$token->setType('test');
		$token->setToken(StringUtils::getRandomString(10));
		$token->setUser($user);

		$em->persist($token);
	}
	$em->flush();
	$em->clear();
	var_dump(memory_get_usage(true));
}

echo "<br>last :".memory_get_usage(true);

result:

first :2097152
int(4194304) int(4194304) int(4194304) int(6291456) int(8388608) 
last :8388608

and then I refresh page, result:

first :2097152
int(2097152) int(2097152) int(2097152) int(2097152) int(4194304) 
last :4194304

after X refreshes in a row, result will be:

first :2097152
int(2097152) int(2097152) int(2097152) int(2097152) int(2097152) 
last :2097152

then ... I'm waiting X second and making another refresh:

first :2097152
int(8388608) int(8388608) int(8388608) int(10485760) int(12582912) 
last :12582912

why is it using different memory every first time ?
when I'm not making flush, memory is STILL SAME -> 2097152
is it bug of UnitOfWork .. or something with apache ? php ? db ?

I have never ending loop for selecting & creating entities ... after 1 hour apache process is using more than 500MB of memory

Originally created by @poolerMF on GitHub (Jan 25, 2018). Originally assigned to: @Ocramius on GitHub. example: echo "first :".memory_get_usage(true)."<br>"; $em = $this->getDoctrine(); for($y = 0;$y<5;$y++) { $user = $em->getRepository(User::class)->find(1); for($x = 0;$x<500+$y;$x++) { $token = new CentralToken(); $token->setType('test'); $token->setToken(StringUtils::getRandomString(10)); $token->setUser($user); $em->persist($token); } $em->flush(); $em->clear(); var_dump(memory_get_usage(true)); } echo "<br>last :".memory_get_usage(true); result: first :2097152 int(4194304) int(4194304) int(4194304) int(6291456) int(8388608) last :8388608 and then I refresh page, result: first :2097152 int(2097152) int(2097152) int(2097152) int(2097152) int(4194304) last :4194304 after X refreshes in a row, result will be: first :2097152 int(2097152) int(2097152) int(2097152) int(2097152) int(2097152) last :2097152 then ... I'm waiting X second and making another refresh: first :2097152 int(8388608) int(8388608) int(8388608) int(10485760) int(12582912) last :12582912 why is it using different memory every first time ? when I'm **not making flush, memory is STILL SAME** -> 2097152 is it bug of UnitOfWork .. or something with apache ? php ? db ? I have never ending loop for selecting & creating entities ... after 1 hour apache process is using more than 500MB of memory
admin added the BugInvalidMissing Tests labels 2026-01-22 15:20:20 +01:00
admin closed this issue 2026-01-22 15:20:20 +01:00
admin commented 2026-01-22 15:20:21 +01:00
Author
Owner
Copy Link

@jvasseur commented on GitHub (Jan 25, 2018):

My guess would be that you have some kind of logger active that keep in memory queries executed by flush.

@jvasseur commented on GitHub (Jan 25, 2018): My guess would be that you have some kind of logger active that keep in memory queries executed by `flush`.
admin commented 2026-01-22 15:20:21 +01:00
Author
Owner
Copy Link

@poolerMF commented on GitHub (Jan 26, 2018):

@jvasseur no I don't have
if I had, memory will be incerased EVERY time ... you see, that memory is not increased every time

I tested it in another NEW project and result is same

@poolerMF commented on GitHub (Jan 26, 2018): @jvasseur no I don't have if I had, memory will be incerased EVERY time ... you see, that memory is not increased every time I tested it in another NEW project and result is same
admin commented 2026-01-22 15:20:22 +01:00
Author
Owner
Copy Link

@Ocramius commented on GitHub (Jan 26, 2018):

Can you try using https://github.com/Ocramius/DoctrineBatchUtils and see if
the problem still occurs for that one loop?

Do you have any listeners or loggers attached to the connection or the
entitymanager?

On 26 Jan 2018 09:12, "poolerMF" notifications@github.com wrote:

@jvasseur https://github.com/jvasseur no I don't have
if I had, memory will be incerased EVERY time ... you see, that memory is
not increased every time

I tested it in another NEW project and result is same

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/doctrine/doctrine2/issues/7009#issuecomment-360712299,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJakOBLufk2hj8iUCPF7fvqLbfP9LUdks5tOYkIgaJpZM4Rs6yY
.

@Ocramius commented on GitHub (Jan 26, 2018): Can you try using https://github.com/Ocramius/DoctrineBatchUtils and see if the problem still occurs for that one loop? Do you have any listeners or loggers attached to the connection or the entitymanager? On 26 Jan 2018 09:12, "poolerMF" <notifications@github.com> wrote: > @jvasseur <https://github.com/jvasseur> no I don't have > if I had, memory will be incerased EVERY time ... you see, that memory is > not increased every time > > I tested it in another NEW project and result is same > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <https://github.com/doctrine/doctrine2/issues/7009#issuecomment-360712299>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAJakOBLufk2hj8iUCPF7fvqLbfP9LUdks5tOYkIgaJpZM4Rs6yY> > . >
admin commented 2026-01-22 15:20:23 +01:00
Author
Owner
Copy Link

@coudenysj commented on GitHub (Jan 26, 2018):

@poolerMF It is a pretty basic approach, but you could dump the entity manager in a file a couple of times, and use diff to figure out what's going on: https://blog.jachim.be/2013/01/finding-memory-leaks-in-php-objects/.

@coudenysj commented on GitHub (Jan 26, 2018): @poolerMF It is a pretty basic approach, but you could dump the entity manager in a file a couple of times, and use `diff` to figure out what's going on: https://blog.jachim.be/2013/01/finding-memory-leaks-in-php-objects/.
admin commented 2026-01-22 15:20:23 +01:00
Author
Owner
Copy Link

@poolerMF commented on GitHub (Jan 26, 2018):

I tried to test it in NEW CLEAN project as I wrote ... and result was same

@Ocramius SimpleBatchIteratorAggregate is doing same thing as me in script ... I'm calling flush and clear after 500 iterations ... I tried It with transaction, but result is same

@coudenysj it's not possible to dump EM or UnitOfWork - apache will get out of memory

  • this occured every time ... even without selecting/persisting something

I need better tool for measuring it ... any ideas ?

@poolerMF commented on GitHub (Jan 26, 2018): I tried to test it in NEW CLEAN project as I wrote ... and result was same @Ocramius SimpleBatchIteratorAggregate is doing same thing as me in script ... I'm calling flush and clear after 500 iterations ... I tried It with transaction, but result is same @coudenysj it's not possible to dump EM or UnitOfWork - apache will get out of memory - this occured every time ... even without selecting/persisting something I need better tool for measuring it ... any ideas ?
admin commented 2026-01-22 15:20:23 +01:00
Author
Owner
Copy Link

@Ocramius commented on GitHub (Jan 26, 2018):

Dumping the keys of the UnitOfWork#identityMap would be sufficient - no need to dump the contents.

@Ocramius commented on GitHub (Jan 26, 2018): Dumping the keys of the `UnitOfWork#identityMap` would be sufficient - no need to dump the contents.
admin commented 2026-01-22 15:20:24 +01:00
Author
Owner
Copy Link

@poolerMF commented on GitHub (Jan 26, 2018):

I tried and identityMap is empty after every EM->clear

I found DebugUnitOfWorkListener .. tried it and result after every flush:
Flush Operation [] - Empty identity map.

but it's OK that identity map is empty, it's normal behaviour after calling EM->clear

@poolerMF commented on GitHub (Jan 26, 2018): I tried and identityMap is empty after every EM->clear I found **DebugUnitOfWorkListener** .. tried it and result after every flush: Flush Operation [] - Empty identity map. but it's OK that identity map is empty, it's normal behaviour after calling EM->clear
admin commented 2026-01-22 15:20:25 +01:00
Author
Owner
Copy Link

@Ocramius commented on GitHub (Jan 26, 2018):

@poolerMF are you running the latest ORM version? Any other properties we can check?

What about the listeners? Anything registered?

@Ocramius commented on GitHub (Jan 26, 2018): @poolerMF are you running the latest ORM version? Any other properties we can check? What about the listeners? Anything registered?
admin commented 2026-01-22 15:20:25 +01:00
Author
Owner
Copy Link

@coudenysj commented on GitHub (Jan 28, 2018):

@poolerMF Can you describe your setup a bit more? It is kind of strange that your memory keeps going up when you're doing page refreshes (as that would be new PHP runs every time).

@coudenysj commented on GitHub (Jan 28, 2018): @poolerMF Can you describe your setup a bit more? It is kind of strange that your memory keeps going up when you're doing page refreshes (as that would be new PHP runs every time).
admin commented 2026-01-22 15:20:25 +01:00
Author
Owner
Copy Link

@poolerMF commented on GitHub (Jan 28, 2018):

@coudenysj look at my first post ...

  • for testing you can use simple entity with 1-3 properties (less variables = you need to run loop more times .. call inner loop 700 ... 1000 times)
  • anything special
  • everything was default in new clean symfony project (v 3.3)
@poolerMF commented on GitHub (Jan 28, 2018): @coudenysj look at my first post ... - for testing you can use simple entity with 1-3 properties (less variables = you need to run loop more times .. call inner loop 700 ... 1000 times) - anything special - everything was default in new clean symfony project (v 3.3)
admin commented 2026-01-22 15:20:26 +01:00
Author
Owner
Copy Link

@coudenysj commented on GitHub (Jan 29, 2018):

@poolerMF Can you push the Symfony project to github somewhere? By using Symfony, you have SQL logging, etc... in the Symfony profiler.

@coudenysj commented on GitHub (Jan 29, 2018): @poolerMF Can you push the Symfony project to github somewhere? By using Symfony, you have SQL logging, etc... in the Symfony profiler.
admin commented 2026-01-22 15:20:26 +01:00
Author
Owner
Copy Link

@poolerMF commented on GitHub (Jan 29, 2018):

class DDC7009Test extends \Doctrine\Tests\OrmFunctionalTestCase
{
	public function setUp()
	{
		parent::setUp();
		try {
			$this->schemaTool->createSchema(
				[
					$this->em->getClassMetadata(DDC7009Anything::class),
				]
			);
		} catch (\Exception $e) {
		}
	}
	public function testIssue()
	{
		$startMemory = memory_get_usage(true);

		for($y = 0;$y<10;$y++) {
			for($x = 0;$x<1000+$y;$x++) {
				$object = new DDC7009Anything();
				$object->setText(uniqid());
				$this->em->persist($object);
			}
			$this->em->flush();
			$this->em->clear();
		}

		$endMemory = memory_get_usage(true);

		self::assertSame($startMemory, $endMemory);
	}
}
/**
 * @ORM\Entity
 */
class DDC7009Anything
{
	/**
	 * @ORM\Id
	 * @ORM\GeneratedValue
	 * @ORM\Column(type="integer")
	 */
	public $id;

	/**
	 * @ORM\Column(name="_text", type="string", length=23)
	 */
	public $text;
}

@coudenysj is it enough for you ?

or you can dump memory in every loop (1-10)

@poolerMF commented on GitHub (Jan 29, 2018): class DDC7009Test extends \Doctrine\Tests\OrmFunctionalTestCase { public function setUp() { parent::setUp(); try { $this->schemaTool->createSchema( [ $this->em->getClassMetadata(DDC7009Anything::class), ] ); } catch (\Exception $e) { } } public function testIssue() { $startMemory = memory_get_usage(true); for($y = 0;$y<10;$y++) { for($x = 0;$x<1000+$y;$x++) { $object = new DDC7009Anything(); $object->setText(uniqid()); $this->em->persist($object); } $this->em->flush(); $this->em->clear(); } $endMemory = memory_get_usage(true); self::assertSame($startMemory, $endMemory); } } /** * @ORM\Entity */ class DDC7009Anything { /** * @ORM\Id * @ORM\GeneratedValue * @ORM\Column(type="integer") */ public $id; /** * @ORM\Column(name="_text", type="string", length=23) */ public $text; } @coudenysj is it enough for you ? or you can dump memory in every loop (1-10)
admin commented 2026-01-22 15:20:27 +01:00
Author
Owner
Copy Link

@coudenysj commented on GitHub (Jan 29, 2018):

@poolerMF I've add the code in a branch on my fork.

First of all, the assertSame can never be the same, because you declare loop variables (which impact the memory usage).

The reason why the memory is increasing, is the SQL Logger in the tests. If you disabled it (like I did in the setUp), you don't get the extra memory usage.

@coudenysj commented on GitHub (Jan 29, 2018): @poolerMF I've add the code in [a branch on my fork](https://github.com/coudenysj/doctrine2/blob/01c9914c2d9453c24e0bc4727f511efa7582272d/tests/Doctrine/Tests/ORM/Functional/Ticket/GH7009Test.php]). First of all, the [assertSame](https://github.com/coudenysj/doctrine2/blob/01c9914c2d9453c24e0bc4727f511efa7582272d/tests/Doctrine/Tests/ORM/Functional/Ticket/GH7009Test.php#L41) can never be the same, because you [declare loop variables](https://github.com/coudenysj/doctrine2/blob/01c9914c2d9453c24e0bc4727f511efa7582272d/tests/Doctrine/Tests/ORM/Functional/Ticket/GH7009Test.php#L29-L30) (which impact the memory usage). The reason why the memory is increasing, is the [SQL Logger in the tests](https://github.com/coudenysj/doctrine2/blob/01c9914c2d9453c24e0bc4727f511efa7582272d/tests/Doctrine/Tests/OrmFunctionalTestCase.php#L66). If you disabled it ([like I did in the setUp](https://github.com/coudenysj/doctrine2/blob/01c9914c2d9453c24e0bc4727f511efa7582272d/tests/Doctrine/Tests/ORM/Functional/Ticket/GH7009Test.php#L13)), you don't get the extra memory usage.
admin commented 2026-01-22 15:20:27 +01:00
Author
Owner
Copy Link

@poolerMF commented on GitHub (Jan 30, 2018):

@coudenysj I didn't test that code, I made it quickly

finnaly I find out, that memory increasing was maybe becouse of SQL logger ... but still it has weird behaviour if using SQL logger (memory is still same ... it should increase every time)

@poolerMF commented on GitHub (Jan 30, 2018): @coudenysj I didn't test that code, I made it quickly finnaly I find out, that memory increasing was maybe becouse of SQL logger ... but still it has weird behaviour if using SQL logger (memory is still same ... it should increase every time)
admin commented 2026-01-22 15:20:27 +01:00
Author
Owner
Copy Link

@Ocramius commented on GitHub (Jan 30, 2018):

Closing here then :)

@Ocramius commented on GitHub (Jan 30, 2018): Closing here then :)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BC Break
Bug
Can't Fix
Deprecation
Documentation
DQL
Duplicate
Failing Test
Hacktoberfest
Improvement
Improvement
Incomplete
Invalid
Missing Tests
New Feature
PR Created
pull-request
Mirrored from GitHub Pull Request
 Question
Regression
Status: Waiting feedback
Waiting feedback
WIP
Won't Fix
No Label Bug Invalid Missing Tests
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
admin
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: doctrine/archived-orm#5861
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?