Many weird characters in metadata and generated sql queries #5749

Closed
opened 2026-01-22 15:16:25 +01:00 by admin · 4 comments
Owner

Originally created by @pylebecq on GitHub (Oct 24, 2017).

Hello guys,

I'm opening an issue because I have on my production servers many issues and I don't know if they are related or not, I don't really know if the issue is coming to from the ORM or the DBAL, and I really can't figure things out, so maybe you guys will have any hint that may help me, or maybe you will be able to point me to another issue I did not find using the search engine that talk about the same kind of issue.

Below you will find many errors that are reported by New Relic with the stack trace (sorry, it will be a long post), and I will explain what I notice. I just can't figure the root cause out. There is one thing for you to know: there are no typo in our code. The issues only appear after some hours, not immediately to add a little more weird to the story.

I'm using Doctrine ORM 2.5.10 inside a Symfony 2.8.27 project, using doctrine bundle 1.6.8. We're running php 7.0.20, and using apcu as a metadata/query/result cache.

ReflectionException: Noticed exception 'ReflectionException' with message 'Class Mdm\Bundle[ProductBundle\Entity\Shop does not exist' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/RuntimeReflectionService.php:72

…ne/common/lib/Doctrine/Common/Persistence/Mapping/RuntimeReflectionService.php (72)
…71019093749/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataInfo.php (924)
…19093749/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php (721)
…ommon/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php (214)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php (281)
…9/vendor/doctrine/orm/lib/Doctrine/ORM/Repository/DefaultRepositoryFactory.php (44)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php (698)
…log-service/releases/20171019093749/app/cache/prod/appProdProjectContainer.php (3162)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2248)
…log-service/releases/20171019093749/app/cache/prod/appProdProjectContainer.php (3072)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2248)
…ymfony/src/Symfony/Component/EventDispatcher/ContainerAwareEventDispatcher.php (183)
…ymfony/src/Symfony/Component/EventDispatcher/ContainerAwareEventDispatcher.php (128)
…ndor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php (45)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3243)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562)
…ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15)

In this one we can see that the class name is wrong, it contains a [ instead of a \: Mdm\Bundle[ProductBundle\Entity\Shop vs Mdm\Bundle\ProductBundle\Entity\Shop

Doctrine\DBAL\Exception\DriverException: Noticed exception 'Doctrine\DBAL\Exception\DriverException' with message 'An exception occurred while executing 'SELECT f0_.id AS id_0, f0_.code AS code_1, f0_.is_publish AS is_publish_2, f1_.text AS text_3, f1_-lang AS lang_4, f1_.id AS id_5 FROM facet f0_ LEFT JOIN facet_translation f1_ ON f0_.id = f1_.id WHERE f1_.lang = ?' with params ["it"]:SQLSTATE[42883]: Undefined function: 7 ERROR: operator does not exist: facet_translation - characterLINE 1: ...s_publish AS is_publish_2, f1_.text AS text_3, f1_-lang AS l... ^HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:91

…leases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php (128)
…/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/in Doctrine\DBAL\DBALException::driverExceptionDuringQuery called at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php
Connection.php (855)
…/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php (899)
…/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php (830)
…93749/vendor/doctrine/orm/lib/Doctrine/ORM/Query/Exec/SingleSelectExecutor.php (50)
…service/releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/Query.php (321)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (969)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (924)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (739)
…es/20171019093749/src/Mdm/Bundle/CategoryBundle/Repository/FacetRepository.php (23)
…/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Services/QueryService.php (380)
…/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (861)
…/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (422)
…/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (308)
…s/20171019093749/src/Mdm/Bundle/ProductBundle/Controller/ProductController.php (739)
in Mdm\Bundle\ProductBundle\Controller\ProductController::searchProducts called at ? (?)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3246)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562)
…ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15)

Here we can see in the SQL query, in the SELECT clause, a reference to f1_-lang instead of f1_.lang

Doctrine\DBAL\Exception\SyntaxErrorException: Noticed exception 'Doctrine\DBAL\Exception\SyntaxErrorException' with message 'An exception occurred while executing 'SELECT f0_.id AS id_0, f0_.code AS code_1, f0_.is_publish AS is_publish_2, f1_.text AS text_3, f1_.lang AS lang_4,f1_.id AS id_5 FROM facet f0_ LEFT JOIN facet_translation f1_ ON f0_.id = f1_.id WHERE f1_.lang = ?' with params ["en"]:SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near ""LINE 1: ...publish_2, f1_.text AS text_3, f1_.lang AS lang_4,f1_.id AS... ^' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:66

…leases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php (128)
…/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php (855)
…93749/vendor/doctrine/orm/lib/Doctrine/ORM/Query/Exec/SingleSelectExecutor.php (50)
…service/releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/Query.php (321)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (969)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (924)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (739)
…es/20171019093749/src/Mdm/Bundle/CategoryBundle/Repository/FacetRepository.php (23)
…/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Services/QueryService.php (380)
…/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (861)
…/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (422)
…/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (308)
…s/20171019093749/src/Mdm/Bundle/ProductBundle/Controller/ProductController.php (739)
in Mdm\Bundle\ProductBundle\Controller\ProductController::searchProducts called at ? (?)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3246)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562)
…ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15)

Here, there is a small weird square just before f1_.id corresponding to a non printable character it seems.

Doctrine\Common\Persistence\Mapping\MappingException: Noticed exception 'Doctrine\Common\Persistence\Mapping\MappingException' with message 'Class 'AppBuncle\Entity\CategoryNewProduct' does not exist' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/MappingException.php:96

…ne/common/lib/Doctrine/Common/Persistence/Mapping/RuntimeReflectionService.php (41)
…ommon/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php (281)
…ommon/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php (311)
…19093749/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php (78)
…ommon/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php (216)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php (281)
…ce/releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php (2510)
…749/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php (273)
…749/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php (494)
…749/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php (165)
…9/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php (150)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (978)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (924)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (727)
…eases/20171019093749/src/AppBundle/Repository/CategoryNewProductRepository.php (25)
…749/src/Mdm/Bundle/ProductBundle/ProductDecorator/NewProductLabelDecorator.php (44)
…19093749/src/Mdm/Bundle/ProductBundle/ProductDecorator/AggregatedDecorator.php (38)
…s/20171019093749/src/Mdm/Bundle/ProductBundle/Controller/ProductController.php (334)
in Mdm\Bundle\ProductBundle\Controller\ProductController::getProductV2 called at ? (?)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3246)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562)
…ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15)

Looks like first example, but the stack trace is not the same. It is still an error in a class name: AppBuncle\Entity\CategoryNewProduct instead of AppBundle\Entity\CategoryNewProduct. This example is very widespread, and it is the same thing every time: a character is replaced by another one, and everytime it is the same pattern: it is replaced by the previous character in the ascii table. In this example, the letter d is replaced by c, I have examples of a ReflectionException concerning a property dmail that does not exist intead of email. If we look the first example, [ is actually preceding \ in the ascii table. I also have another example, still matching the pattern: Mdm\Bundle\C`tegoryBundle\Repository instead of Mdm\Bundle\CategoryBundle\Repository

Doctrine\DBAL\Exception\InvalidFieldNameException: Noticed exception 'Doctrine\DBAL\Exception\InvalidFieldNameException' with message 'An exception occurred while executing 'SELECT s0_.id AS id_0, s0_.shopid AS shopid_1, s0_.market @S market_2, s0_.googletrackingcode AS googletrackingcode_3 FROM shop s0_ WHERE s0_.shopid IN ('00000999', '00000190')':SQLSTATE[42703]: Undefined column: 7 ERROR: column "s" does not existLINE 1: ..._.id AS id_0, s0_.shopid AS shopid_1, s0_.market @S market_2... ^' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:72

I don't even know what to think of this one. We can see in the select clause there is s0_.market @S market_2 instead of s0_.market AS market_2

Doctrine\DBAL\Exception\TableNotFoundException: Noticed exception 'Doctrine\DBAL\Exception\TableNotFoundException' with message 'An exception occurred while executing 'SELECT t0_.id AS id_0, t0_.shopid AS shopid_1, t0_.market AS market_2, t0_.googletrackingcode AS googletrackingcode_3 FROM t0_ WHERE t0_.shopid = ?' with params ["00000190"]:SQLSTATE[42P01]: Undefined table: 7 ERROR: relation "t0_" does not existLINE 1: ....googletrackingcode AS googletrackingcode_3 FROM t0_ WHERE ... ^' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:75

…leases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php (128)
…/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php (855)
…93749/vendor/doctrine/orm/lib/Doctrine/ORM/Query/Exec/SingleSelectExecutor.php (50)
…service/releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/Query.php (321)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (969)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (924)
…releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (766)
…ases/20171019093749/src/Mdm/Bundle/ProductBundle/Repository/ShopRepository.php (37)
…e/releases/20171019093749/src/Mdm/Bundle/ProductBundle/Manager/ShopManager.php (30)
…s/20171019093749/src/Mdm/Bundle/ProductBundle/Controller/ProductController.php (316)
in Mdm\Bundle\ProductBundle\Controller\ProductController::getProductV2 called at ? (?)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3246)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359)
…mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562)
…ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15)

This one is also very weird, the table name used in the FROM clause is the alias instead of the real table name.

This is it. Any help will be greatly appreciated, any hint or anything. Again, I don't know if this is a bug from doctrine or something else but maybe you guys will be able to clarify this. Unfortunately, since the issues come over time, I cannot give you a git repository with some code to reproduce this.

Best regards,

Originally created by @pylebecq on GitHub (Oct 24, 2017). Hello guys, I'm opening an issue because I have on my production servers many issues and I don't know if they are related or not, I don't really know if the issue is coming to from the ORM or the DBAL, and I really can't figure things out, so maybe you guys will have any hint that may help me, or maybe you will be able to point me to another issue I did not find using the search engine that talk about the same kind of issue. Below you will find many errors that are reported by New Relic with the stack trace (sorry, it will be a long post), and I will explain what I notice. I just can't figure the root cause out. There is one thing for you to know: there are no typo in our code. The issues only appear after some hours, not immediately to add a little more weird to the story. I'm using Doctrine ORM 2.5.10 inside a Symfony 2.8.27 project, using doctrine bundle 1.6.8. We're running php 7.0.20, and using apcu as a metadata/query/result cache. ``` ReflectionException: Noticed exception 'ReflectionException' with message 'Class Mdm\Bundle[ProductBundle\Entity\Shop does not exist' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/RuntimeReflectionService.php:72 …ne/common/lib/Doctrine/Common/Persistence/Mapping/RuntimeReflectionService.php (72) …71019093749/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataInfo.php (924) …19093749/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php (721) …ommon/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php (214) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php (281) …9/vendor/doctrine/orm/lib/Doctrine/ORM/Repository/DefaultRepositoryFactory.php (44) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php (698) …log-service/releases/20171019093749/app/cache/prod/appProdProjectContainer.php (3162) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2248) …log-service/releases/20171019093749/app/cache/prod/appProdProjectContainer.php (3072) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2248) …ymfony/src/Symfony/Component/EventDispatcher/ContainerAwareEventDispatcher.php (183) …ymfony/src/Symfony/Component/EventDispatcher/ContainerAwareEventDispatcher.php (128) …ndor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php (45) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3243) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562) …ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15) ``` In this one we can see that the class name is wrong, it contains a **[** instead of a **\\**: **Mdm\Bundle[ProductBundle\Entity\Shop** vs **Mdm\Bundle\ProductBundle\Entity\Shop** ``` Doctrine\DBAL\Exception\DriverException: Noticed exception 'Doctrine\DBAL\Exception\DriverException' with message 'An exception occurred while executing 'SELECT f0_.id AS id_0, f0_.code AS code_1, f0_.is_publish AS is_publish_2, f1_.text AS text_3, f1_-lang AS lang_4, f1_.id AS id_5 FROM facet f0_ LEFT JOIN facet_translation f1_ ON f0_.id = f1_.id WHERE f1_.lang = ?' with params ["it"]:SQLSTATE[42883]: Undefined function: 7 ERROR: operator does not exist: facet_translation - characterLINE 1: ...s_publish AS is_publish_2, f1_.text AS text_3, f1_-lang AS l... ^HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:91 …leases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php (128) …/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/in Doctrine\DBAL\DBALException::driverExceptionDuringQuery called at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php Connection.php (855) …/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php (899) …/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php (830) …93749/vendor/doctrine/orm/lib/Doctrine/ORM/Query/Exec/SingleSelectExecutor.php (50) …service/releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/Query.php (321) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (969) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (924) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (739) …es/20171019093749/src/Mdm/Bundle/CategoryBundle/Repository/FacetRepository.php (23) …/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Services/QueryService.php (380) …/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (861) …/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (422) …/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (308) …s/20171019093749/src/Mdm/Bundle/ProductBundle/Controller/ProductController.php (739) in Mdm\Bundle\ProductBundle\Controller\ProductController::searchProducts called at ? (?) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3246) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562) …ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15) ``` Here we can see in the SQL query, in the SELECT clause, a reference to **f1_-lang** instead of **f1_.lang** ``` Doctrine\DBAL\Exception\SyntaxErrorException: Noticed exception 'Doctrine\DBAL\Exception\SyntaxErrorException' with message 'An exception occurred while executing 'SELECT f0_.id AS id_0, f0_.code AS code_1, f0_.is_publish AS is_publish_2, f1_.text AS text_3, f1_.lang AS lang_4,f1_.id AS id_5 FROM facet f0_ LEFT JOIN facet_translation f1_ ON f0_.id = f1_.id WHERE f1_.lang = ?' with params ["en"]:SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near ""LINE 1: ...publish_2, f1_.text AS text_3, f1_.lang AS lang_4,f1_.id AS... ^' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:66 …leases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php (128) …/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php (855) …93749/vendor/doctrine/orm/lib/Doctrine/ORM/Query/Exec/SingleSelectExecutor.php (50) …service/releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/Query.php (321) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (969) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (924) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (739) …es/20171019093749/src/Mdm/Bundle/CategoryBundle/Repository/FacetRepository.php (23) …/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Services/QueryService.php (380) …/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (861) …/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (422) …/releases/20171019093749/src/Mdm/Bundle/SearchBundle/Manager/SearchManager.php (308) …s/20171019093749/src/Mdm/Bundle/ProductBundle/Controller/ProductController.php (739) in Mdm\Bundle\ProductBundle\Controller\ProductController::searchProducts called at ? (?) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3246) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562) …ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15) ``` Here, there is a small weird square just before `f1_.id` corresponding to a non printable character it seems. ``` Doctrine\Common\Persistence\Mapping\MappingException: Noticed exception 'Doctrine\Common\Persistence\Mapping\MappingException' with message 'Class 'AppBuncle\Entity\CategoryNewProduct' does not exist' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/MappingException.php:96 …ne/common/lib/Doctrine/Common/Persistence/Mapping/RuntimeReflectionService.php (41) …ommon/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php (281) …ommon/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php (311) …19093749/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php (78) …ommon/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php (216) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php (281) …ce/releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php (2510) …749/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php (273) …749/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php (494) …749/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php (165) …9/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php (150) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (978) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (924) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (727) …eases/20171019093749/src/AppBundle/Repository/CategoryNewProductRepository.php (25) …749/src/Mdm/Bundle/ProductBundle/ProductDecorator/NewProductLabelDecorator.php (44) …19093749/src/Mdm/Bundle/ProductBundle/ProductDecorator/AggregatedDecorator.php (38) …s/20171019093749/src/Mdm/Bundle/ProductBundle/Controller/ProductController.php (334) in Mdm\Bundle\ProductBundle\Controller\ProductController::getProductV2 called at ? (?) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3246) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562) …ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15) ``` Looks like first example, but the stack trace is not the same. It is still an error in a class name: **AppBuncle\Entity\CategoryNewProduct** instead of **AppBundle\Entity\CategoryNewProduct**. This example is very widespread, and it is the same thing every time: a character is replaced by another one, and everytime it is the same pattern: it is replaced by the previous character in the ascii table. In this example, the letter **d** is replaced by **c**, I have examples of a ReflectionException concerning a property **dmail** that does not exist intead of **email**. If we look the first example, **[** is actually preceding **\\** in the ascii table. I also have another example, still matching the pattern: **Mdm\Bundle\C`tegoryBundle\Repository** instead of **Mdm\Bundle\CategoryBundle\Repository** ``` Doctrine\DBAL\Exception\InvalidFieldNameException: Noticed exception 'Doctrine\DBAL\Exception\InvalidFieldNameException' with message 'An exception occurred while executing 'SELECT s0_.id AS id_0, s0_.shopid AS shopid_1, s0_.market @S market_2, s0_.googletrackingcode AS googletrackingcode_3 FROM shop s0_ WHERE s0_.shopid IN ('00000999', '00000190')':SQLSTATE[42703]: Undefined column: 7 ERROR: column "s" does not existLINE 1: ..._.id AS id_0, s0_.shopid AS shopid_1, s0_.market @S market_2... ^' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:72 ``` I don't even know what to think of this one. We can see in the select clause there is **s0_.market @S market_2** instead of **s0_.market AS market_2** ``` Doctrine\DBAL\Exception\TableNotFoundException: Noticed exception 'Doctrine\DBAL\Exception\TableNotFoundException' with message 'An exception occurred while executing 'SELECT t0_.id AS id_0, t0_.shopid AS shopid_1, t0_.market AS market_2, t0_.googletrackingcode AS googletrackingcode_3 FROM t0_ WHERE t0_.shopid = ?' with params ["00000190"]:SQLSTATE[42P01]: Undefined table: 7 ERROR: relation "t0_" does not existLINE 1: ....googletrackingcode AS googletrackingcode_3 FROM t0_ WHERE ... ^' in /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:75 …leases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php (128) …/releases/20171019093749/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php (855) …93749/vendor/doctrine/orm/lib/Doctrine/ORM/Query/Exec/SingleSelectExecutor.php (50) …service/releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/Query.php (321) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (969) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (924) …releases/20171019093749/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php (766) …ases/20171019093749/src/Mdm/Bundle/ProductBundle/Repository/ShopRepository.php (37) …e/releases/20171019093749/src/Mdm/Bundle/ProductBundle/Manager/ShopManager.php (30) …s/20171019093749/src/Mdm/Bundle/ProductBundle/Controller/ProductController.php (316) in Mdm\Bundle\ProductBundle\Controller\ProductController::getProductV2 called at ? (?) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3246) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3205) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (3359) …mdm/deploy/mdm-catalog-service/releases/20171019093749/app/bootstrap.php.cache (2562) …ed at /data/mdm/deploy/mdm-catalog-service/releases/20171019093749/web/app.php (15) ``` This one is also very weird, the table name used in the FROM clause is the alias instead of the real table name. This is it. Any help will be greatly appreciated, any hint or anything. Again, I don't know if this is a bug from doctrine or something else but maybe you guys will be able to clarify this. Unfortunately, since the issues come over time, I cannot give you a git repository with some code to reproduce this. Best regards,
admin closed this issue 2026-01-22 15:16:25 +01:00
Author
Owner

@Ocramius commented on GitHub (Oct 24, 2017):

using apcu as a metadata/query/result cache.

Possibly one of the issues here.

…log-service/releases/20171019093749/app/cache/prod/appProdProjectContainer.php (3072)

This bit looks interesting: is it a simple service lookup? What is being passed to EntityManager#getRepository()?

@Ocramius commented on GitHub (Oct 24, 2017): > using apcu as a metadata/query/result cache. Possibly one of the issues here. ``` …log-service/releases/20171019093749/app/cache/prod/appProdProjectContainer.php (3072) ``` This bit looks interesting: is it a simple service lookup? What is being passed to `EntityManager#getRepository()`?
Author
Owner

@pylebecq commented on GitHub (Oct 26, 2017):

Hello @Ocramius ,

Thank you for taking some time to review all of this.

In the appProdProjectContainer.php at line 3072 there is this method:

protected function getMdm_Requestlistener_SrvService()
{
    return $this->services['mdm.requestlistener.srv'] = new \Mdm\Service\MdmRequestListener($this->get('doctrine.orm.legacy_entity_manager'), $this->get('mdm.shop.repository'));
}

And the $this->get('mdm.shop.repository') results to a call to this other method in the appProdProjectContainer:

protected function getMdm_Shop_RepositoryService()
{
    return $this->services['mdm.shop.repository'] = $this->get('doctrine.orm.symfony_entity_manager')->getRepository('Mdm\\Bundle\\ProductBundle\\Entity\\Shop');
}

About APCu, for now we are trying to reduce its usage by removing the result cache from it, it seems the servers are becoming more stable. I opened an issue because I wanted to find the root cause to make sure we won't run into it again in the future.

@pylebecq commented on GitHub (Oct 26, 2017): Hello @Ocramius , Thank you for taking some time to review all of this. In the appProdProjectContainer.php at line 3072 there is this method: ```php protected function getMdm_Requestlistener_SrvService() { return $this->services['mdm.requestlistener.srv'] = new \Mdm\Service\MdmRequestListener($this->get('doctrine.orm.legacy_entity_manager'), $this->get('mdm.shop.repository')); } ``` And the `$this->get('mdm.shop.repository')` results to a call to this other method in the appProdProjectContainer: ```php protected function getMdm_Shop_RepositoryService() { return $this->services['mdm.shop.repository'] = $this->get('doctrine.orm.symfony_entity_manager')->getRepository('Mdm\\Bundle\\ProductBundle\\Entity\\Shop'); } ``` About APCu, for now we are trying to reduce its usage by removing the result cache from it, it seems the servers are becoming more stable. I opened an issue because I wanted to find the root cause to make sure we won't run into it again in the future.
Author
Owner

@Majkl578 commented on GitHub (Dec 19, 2017):

There is no error in the container code provided, entity class' name is ok as well.

In third error log, it would be interesting what the square sign is, but I don't think it would significantly help (unless it's \x00, APC used to have problem with nulls).
In last error log the table name is completely missing.

This doesn't look like something ORM itself would cause, I would definitely blame APCu or other middleware. As far as I know APCu doesn't use any locking mechanism and doesn't have any sort of transactions, couldn't this be caused by high load & number of writes? But anyway, I'd suggest to switch to different storage, i.e. Redis.

@Majkl578 commented on GitHub (Dec 19, 2017): There is no error in the container code provided, entity class' name is ok as well. In third error log, it would be interesting what the _square sign_ is, but I don't think it would significantly help (unless it's \x00, APC used to have problem with nulls). In last error log the table name is completely missing. This doesn't look like something ORM itself would cause, I would definitely blame APCu or other middleware. As far as I know APCu doesn't use any locking mechanism and doesn't have any sort of transactions, couldn't this be caused by high load & number of writes? But anyway, I'd suggest to switch to different storage, i.e. Redis.
Author
Owner

@pylebecq commented on GitHub (Dec 20, 2017):

Yes, it seems APCu is the one to blame here. We switched the result cache to redis, it's working fine now. Metadata and Query cache are still using APCu.

I don't really know what happens to be honest. There is a lot to read about APCu cache fragmentation, I don't know if it takes part in this issue or not but for now we have removed all usage of TTL with APCu and the issue does no happen anymore. It seems like add/removal of cache entries "corrupts" other entries eventually.

Thank you guys for taking time reading my long original post and giving me a few hints and places to look for.

@pylebecq commented on GitHub (Dec 20, 2017): Yes, it seems APCu is the one to blame here. We switched the result cache to redis, it's working fine now. Metadata and Query cache are still using APCu. I don't really know what happens to be honest. There is a lot to read about APCu cache fragmentation, I don't know if it takes part in this issue or not but for now we have removed all usage of TTL with APCu and the issue does no happen anymore. It seems like add/removal of cache entries "corrupts" other entries eventually. Thank you guys for taking time reading my long original post and giving me a few hints and places to look for.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: doctrine/archived-orm#5749