doctrine/archived-orm
1
0
Fork 0
mirror of https://github.com/doctrine/orm.git synced 2026-03-23 22:42:18 +01:00
Code Issues 5.3k Packages Projects Releases 100 Wiki Activity

[PR #580] [CLOSED] [WIP] Second level cache #8402

New Issue
Open
opened 2026-01-22 15:59:45 +01:00 by admin · 0 comments
admin commented 2026-01-22 15:59:45 +01:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/doctrine/orm/pull/580
Author: @FabioBatSilva
Created: 2/14/2013
Status: ❌ Closed

Base: master ← Head: second-level-cache

📝 Commits (10+)

  • 8fe90d1 Second cache level POC
  • 52d4b88 First round of refactory
  • 55146a3 Fix factory method name
  • 0731254 refactoring region access tests
  • 9466a44 basically support for many to many collection
  • d532fdf Refactoring tests
  • 3468ed7 store cascade collections
  • b062340 refactoring collections store
  • 2904ecf handle one to many collections storage
  • 0ee0833 tests put entities on persist

📊 Changes

141 files changed (+14619 additions, -381 deletions)

View changed files

📝 .travis.yml (+7 -4)
📝 docs/en/index.rst (+1 -0)
➕ docs/en/reference/second-level-cache.rst (+802 -0)
📝 docs/en/toc.rst (+8 -4)
📝 doctrine-mapping.xsd (+18 -0)
📝 lib/Doctrine/ORM/AbstractQuery.php (+220 -8)
➕ lib/Doctrine/ORM/Cache.php (+185 -0)
➕ lib/Doctrine/ORM/Cache/CacheEntry.php (+32 -0)
➕ lib/Doctrine/ORM/Cache/CacheException.php (+72 -0)
➕ lib/Doctrine/ORM/Cache/CacheFactory.php (+95 -0)
➕ lib/Doctrine/ORM/Cache/CacheKey.php (+36 -0)
➕ lib/Doctrine/ORM/Cache/CollectionCacheEntry.php (+51 -0)
➕ lib/Doctrine/ORM/Cache/CollectionCacheKey.php (+60 -0)
➕ lib/Doctrine/ORM/Cache/CollectionHydrator.php (+54 -0)
➕ lib/Doctrine/ORM/Cache/ConcurrentRegion.php (+59 -0)
➕ lib/Doctrine/ORM/Cache/DefaultCache.php (+346 -0)
➕ lib/Doctrine/ORM/Cache/DefaultCacheFactory.php (+193 -0)
➕ lib/Doctrine/ORM/Cache/DefaultCollectionHydrator.php (+103 -0)
➕ lib/Doctrine/ORM/Cache/DefaultEntityHydrator.php (+151 -0)
➕ lib/Doctrine/ORM/Cache/DefaultQueryCache.php (+294 -0)

...and 80 more files

📄 Description

Hi guys. :)

After a look into some implementations I end up with the following solution for the second level cache..

There is lot of work todo before merge it, but i'd like to get your thoughts before i go any further on this approach.
I hope my drafts are good enough to explain the idea :

Cache strategies

* READ_ONLY (DEFAULT)   : ReadOnly cache can do reads, inserts and deletes, cannot perform updates or employ any locks.
* NONSTRICT_READ_WRITE  : Nonstrict Read Write Cache doesn’t employ any locks but can do reads, inserts , updates and deletes.
* READ_WRITE            : Read Write cache employs locks the entity before update/delete.

classes / interfaces

  • Region :
    Defines a contract for accessing a entity/collection data cache. (Doesn’t employ any locks)
  • ConcurrentRegion :
    Defines contract for concurrently managed data region. (Locks the data before update/delete.)
  • CacheKey / EntityCacheKey / CollectionCacheKey/ QueryCacheKey:
    Defines entity / collection key to be stored in the cache region.
  • EntityHidrator / CollectionHidrator
    Build cache entries and rebuild entities/colection from cache
  • CacheFactory
    Factory from second level cache components

Collection Caching

The most common use case is to cache entities. But we can also cache relationships.
A “collection cache” caches the primary keys of entities that are members of a collection (OneToMany/ManyToMany).
and each element will be cached into its region.

Only identifiers will be cached for collection. When a collection is read from the second level cache it will create proxies based on the cached identifiers, if the application needs to access an element, Doctrine will go to the cache to load the element data.

Query Cache

The query cache does not cache the state of the actual entities in the result set;
it caches only identifier values for an individual query.
So the query cache should always be used in conjunction with the second-level cache.

Query Cache validation

UpdateTimestampsCacheRegion (hibernate approach) :
  • The timestamp cache region keeps track of the last update for each table. (updated for each table modification)
  • A single timestamps region it's utilized by all query cache instances.
  • In most hibernate cache implementations is recommend do not configured cache timeout at all.
  • When a query is loaded from cache, the timestamp region is checked for all tables in the query.
  • If the timestamp of the last update on a table is greater than the time the query results were cached,
    Then the entry is removed and the query goes straight to the database.

OPERATIONS

INSERT :

*************************************************************************************************
UnitOfWork#commit
    Connection#beginTransaction
    Persister#executeInserts
    Connection#commit
    CachedPersister#afterTransactionComplete
        -> Region#put
*************************************************************************************************
                    | READ-ONLY             | NONSTRICT-READ-WRITE      | READ-WRITE            |
-------------------------------------------------------------------------------------------------
pre-insert          |                       |                           |                       |
-------------------------------------------------------------------------------------------------
on-insert           |                       |                           |                       |
-------------------------------------------------------------------------------------------------
after-transaction   | add item to the cache | add item to the cache     | add item to the cache |
-------------------------------------------------------------------------------------------------

UPDATE :

*************************************************************************************************
UnitOfWork#commit
    Connection#beginTransaction
    CachedPersister#update
        -> Region#lock
        -> execute
    Connection#commit
    CachedPersister#afterTransactionComplete
        -> Region#put
        -> Region#unlock
*************************************************************************************************
                    | READ-ONLY             | NONSTRICT-READ-WRITE      | READ-WRITE            |
-------------------------------------------------------------------------------------------------
pre-update          |                       |                           | lock item             |
-------------------------------------------------------------------------------------------------
on-update           | throws exception      |                           |                       |
-------------------------------------------------------------------------------------------------
after-transaction   |                       |  update item cache        | remove item cache     |
-------------------------------------------------------------------------------------------------

DELETE :

*************************************************************************************************
UnitOfWork#commit
    Connection#beginTransaction
    CachedPersister#delete
        -> Region#lock
        -> execute
    Connection#commit
    CachedPersister#afterTransactionComplete
        -> Region#evict
*************************************************************************************************
                    | READ-ONLY             | NONSTRICT-READ-WRITE      | READ-WRITE            |
-------------------------------------------------------------------------------------------------
pre-remove          |                       |                           |                       |
-------------------------------------------------------------------------------------------------
on-remove           |                       |                           | lock item             |
-------------------------------------------------------------------------------------------------
after-transaction   | remove item cache     |  remove item cache        | remove item cache     |
-------------------------------------------------------------------------------------------------

USAGE :

<?php

/**
 * @Entity
 * @Cache("NONSTRICT_READ_WRITE")
 */
class State
{
    /**
     * @Id
     * @GeneratedValue
     * @Column(type="integer")
     */
    protected $id;
    /**
     * @Column
     */
    protected $name;
    /**
     * @Cache()
     * @ManyToOne(targetEntity="Country")
     * @JoinColumn(name="country_id", referencedColumnName="id")
     */
    protected $country;
    /**
     * @Cache()
     * @OneToMany(targetEntity="City", mappedBy="state")
     */
    protected $cities;
}

<?php

$em->persist(new State($name, $country));
$em->flush();                                // Put into cache

$em->clear();                                // Clear entity manager

$state   = $em->find('Entity\State', 1);     // Retreive item from cache
$country = $state->getCountry();             // Retreive item from cache
$cities  = $state->getCities();              // Load from database and put into cache

$state->setName("New Name");
$em->persist($state);
$em->flush();                                // Update item cache

$em->clear();                                // Clear entity manager

$em->find('Entity\State', 1)->getCities();   // Retreive from cache


$em->getCache()->containsEntity('Entity\State', $state->getId())  // Check if the cache exists
$em->getCache()->evictEntity('Entity\State', $state->getId());    // Remove an entity from cache
$em->getCache()->evictEntityRegion('Entity\State');               // Remove all entities from cache

$em->getCache()->containsCollection('Entity\State', 'cities', $state->getId());   // Check if the cache exists        
$em->getCache()->evictCollection('Entity\State', 'cities', $state->getId());      // Remove an entity collection from cache
$em->getCache()->evictCollectionRegion('Entity\State', 'cities');                 // Remove all collections from cache

TODO :

  • Handle many to many collection
  • Handle inheritance
  • Remove/add colection items on update
  • Improve region tests
  • Improve cached persisters coverage
  • Implement xml / yml / php drivers
  • Implement transaction region (Improve cache drivers)
  • Implement query cache (need more tests)
  • Read and write region when using query cache
  • Update documentation
  • Implement cache cache hits and misses log
  • .... ????

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/doctrine/orm/pull/580 **Author:** [@FabioBatSilva](https://github.com/FabioBatSilva) **Created:** 2/14/2013 **Status:** ❌ Closed **Base:** `master` ← **Head:** `second-level-cache` --- ### 📝 Commits (10+) - [`8fe90d1`](https://github.com/doctrine/orm/commit/8fe90d1fbbc775229349fe866bd1d64cfa55203a) Second cache level POC - [`52d4b88`](https://github.com/doctrine/orm/commit/52d4b88ebcd0e15e0d6765db85e6b81c1d7133a2) First round of refactory - [`55146a3`](https://github.com/doctrine/orm/commit/55146a357a881cc61711e25046c3c1ec392543c9) Fix factory method name - [`0731254`](https://github.com/doctrine/orm/commit/0731254558cac5477a970ecdb060b8c0c5482a34) refactoring region access tests - [`9466a44`](https://github.com/doctrine/orm/commit/9466a44a9dd75a69420185fee9449542ee88456e) basically support for many to many collection - [`d532fdf`](https://github.com/doctrine/orm/commit/d532fdf04dc6044f390f64543deda95dd0f8b827) Refactoring tests - [`3468ed7`](https://github.com/doctrine/orm/commit/3468ed724423f1382c8bcfa417909a66bc30bb74) store cascade collections - [`b062340`](https://github.com/doctrine/orm/commit/b0623406a43f9d8e8994161242d3735296b1af23) refactoring collections store - [`2904ecf`](https://github.com/doctrine/orm/commit/2904ecfa29e80c61a7086fe161f56c3c984dfc6f) handle one to many collections storage - [`0ee0833`](https://github.com/doctrine/orm/commit/0ee0833b7fa6a5e59adb0ab957126933f295e2dc) tests put entities on persist ### 📊 Changes **141 files changed** (+14619 additions, -381 deletions) <details> <summary>View changed files</summary> 📝 `.travis.yml` (+7 -4) 📝 `docs/en/index.rst` (+1 -0) ➕ `docs/en/reference/second-level-cache.rst` (+802 -0) 📝 `docs/en/toc.rst` (+8 -4) 📝 `doctrine-mapping.xsd` (+18 -0) 📝 `lib/Doctrine/ORM/AbstractQuery.php` (+220 -8) ➕ `lib/Doctrine/ORM/Cache.php` (+185 -0) ➕ `lib/Doctrine/ORM/Cache/CacheEntry.php` (+32 -0) ➕ `lib/Doctrine/ORM/Cache/CacheException.php` (+72 -0) ➕ `lib/Doctrine/ORM/Cache/CacheFactory.php` (+95 -0) ➕ `lib/Doctrine/ORM/Cache/CacheKey.php` (+36 -0) ➕ `lib/Doctrine/ORM/Cache/CollectionCacheEntry.php` (+51 -0) ➕ `lib/Doctrine/ORM/Cache/CollectionCacheKey.php` (+60 -0) ➕ `lib/Doctrine/ORM/Cache/CollectionHydrator.php` (+54 -0) ➕ `lib/Doctrine/ORM/Cache/ConcurrentRegion.php` (+59 -0) ➕ `lib/Doctrine/ORM/Cache/DefaultCache.php` (+346 -0) ➕ `lib/Doctrine/ORM/Cache/DefaultCacheFactory.php` (+193 -0) ➕ `lib/Doctrine/ORM/Cache/DefaultCollectionHydrator.php` (+103 -0) ➕ `lib/Doctrine/ORM/Cache/DefaultEntityHydrator.php` (+151 -0) ➕ `lib/Doctrine/ORM/Cache/DefaultQueryCache.php` (+294 -0) _...and 80 more files_ </details> ### 📄 Description Hi guys. :) After a look into some implementations I end up with the following solution for the second level cache.. There is lot of work todo before merge it, but i'd like to get your thoughts before i go any further on this approach. I hope my drafts are good enough to explain the idea : ### Cache strategies ``` * READ_ONLY (DEFAULT) : ReadOnly cache can do reads, inserts and deletes, cannot perform updates or employ any locks. * NONSTRICT_READ_WRITE : Nonstrict Read Write Cache doesn’t employ any locks but can do reads, inserts , updates and deletes. * READ_WRITE : Read Write cache employs locks the entity before update/delete. ``` ### classes / interfaces - **Region** : Defines a contract for accessing a entity/collection data cache. (Doesn’t employ any locks) - **ConcurrentRegion** : Defines contract for concurrently managed data region. (Locks the data before update/delete.) - **CacheKey / EntityCacheKey / CollectionCacheKey/ QueryCacheKey**: Defines entity / collection key to be stored in the cache region. - **EntityHidrator / CollectionHidrator** Build cache entries and rebuild entities/colection from cache - **CacheFactory** Factory from second level cache components ### Collection Caching The most common use case is to cache entities. But we can also cache relationships. A “collection cache” caches the primary keys of entities that are members of a collection (OneToMany/ManyToMany). and each element will be cached into its region. Only identifiers will be cached for collection. When a collection is read from the second level cache it will create proxies based on the cached identifiers, if the application needs to access an element, Doctrine will go to the cache to load the element data. ### Query Cache The query cache does not cache the state of the actual entities in the result set; it caches only identifier values for an individual query. So the query cache should always be used in conjunction with the second-level cache. #### Query Cache validation > ##### UpdateTimestampsCacheRegion (hibernate approach) : > - The timestamp cache region keeps track of the last update for each table. (updated for each table modification) > - A single timestamps region it's utilized by all query cache instances. > - In most hibernate cache implementations is recommend do not configured cache timeout at all. > - When a query is loaded from cache, the timestamp region is checked for all tables in the query. > - If the timestamp of the last update on a table is greater than the time the query results were cached, > Then the entry is removed and the query goes straight to the database. ### OPERATIONS #### INSERT : ``` ************************************************************************************************* UnitOfWork#commit Connection#beginTransaction Persister#executeInserts Connection#commit CachedPersister#afterTransactionComplete -> Region#put ************************************************************************************************* | READ-ONLY | NONSTRICT-READ-WRITE | READ-WRITE | ------------------------------------------------------------------------------------------------- pre-insert | | | | ------------------------------------------------------------------------------------------------- on-insert | | | | ------------------------------------------------------------------------------------------------- after-transaction | add item to the cache | add item to the cache | add item to the cache | ------------------------------------------------------------------------------------------------- ``` #### UPDATE : ``` ************************************************************************************************* UnitOfWork#commit Connection#beginTransaction CachedPersister#update -> Region#lock -> execute Connection#commit CachedPersister#afterTransactionComplete -> Region#put -> Region#unlock ************************************************************************************************* | READ-ONLY | NONSTRICT-READ-WRITE | READ-WRITE | ------------------------------------------------------------------------------------------------- pre-update | | | lock item | ------------------------------------------------------------------------------------------------- on-update | throws exception | | | ------------------------------------------------------------------------------------------------- after-transaction | | update item cache | remove item cache | ------------------------------------------------------------------------------------------------- ``` #### DELETE : ``` ************************************************************************************************* UnitOfWork#commit Connection#beginTransaction CachedPersister#delete -> Region#lock -> execute Connection#commit CachedPersister#afterTransactionComplete -> Region#evict ************************************************************************************************* | READ-ONLY | NONSTRICT-READ-WRITE | READ-WRITE | ------------------------------------------------------------------------------------------------- pre-remove | | | | ------------------------------------------------------------------------------------------------- on-remove | | | lock item | ------------------------------------------------------------------------------------------------- after-transaction | remove item cache | remove item cache | remove item cache | ------------------------------------------------------------------------------------------------- ``` #### USAGE : ``` php <?php /** * @Entity * @Cache("NONSTRICT_READ_WRITE") */ class State { /** * @Id * @GeneratedValue * @Column(type="integer") */ protected $id; /** * @Column */ protected $name; /** * @Cache() * @ManyToOne(targetEntity="Country") * @JoinColumn(name="country_id", referencedColumnName="id") */ protected $country; /** * @Cache() * @OneToMany(targetEntity="City", mappedBy="state") */ protected $cities; } ``` ``` php <?php $em->persist(new State($name, $country)); $em->flush(); // Put into cache $em->clear(); // Clear entity manager $state = $em->find('Entity\State', 1); // Retreive item from cache $country = $state->getCountry(); // Retreive item from cache $cities = $state->getCities(); // Load from database and put into cache $state->setName("New Name"); $em->persist($state); $em->flush(); // Update item cache $em->clear(); // Clear entity manager $em->find('Entity\State', 1)->getCities(); // Retreive from cache $em->getCache()->containsEntity('Entity\State', $state->getId()) // Check if the cache exists $em->getCache()->evictEntity('Entity\State', $state->getId()); // Remove an entity from cache $em->getCache()->evictEntityRegion('Entity\State'); // Remove all entities from cache $em->getCache()->containsCollection('Entity\State', 'cities', $state->getId()); // Check if the cache exists $em->getCache()->evictCollection('Entity\State', 'cities', $state->getId()); // Remove an entity collection from cache $em->getCache()->evictCollectionRegion('Entity\State', 'cities'); // Remove all collections from cache ``` #### TODO : - [x] Handle many to many collection - [x] Handle inheritance - [x] Remove/add colection items on update - [x] Improve region tests - [x] Improve cached persisters coverage - [x] Implement xml / yml / php drivers - [x] Implement transaction region _(Improve cache drivers)_ - [x] Implement query cache _**(need more tests)**_ - [x] Read and write region when using query cache - [x] Update documentation - [x] Implement cache cache hits and misses log - [ ] .... ???? --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
admin added the pull-request label 2026-01-22 15:59:45 +01:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BC Break
Bug
Can't Fix
Deprecation
Documentation
DQL
Duplicate
Failing Test
Hacktoberfest
Improvement
Improvement
Incomplete
Invalid
Missing Tests
New Feature
PR Created
pull-request
Mirrored from GitHub Pull Request
 Question
Regression
Status: Waiting feedback
Waiting feedback
WIP
Won't Fix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
admin
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: doctrine/archived-orm#8402
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?