If you are using Caching with groups, Cache::clearGroup will clear all the cache groups not just the one you are trying to clear

I’m using database cache since cakephp 2. From time to time I had trouble with caching using groups. My idea is that every table in the database has it’s own cache group, and therefore I can control when do I want to clear the cache for specific tables. I have a behavior looks like this:

protected function cacheKey(SelectQuery $query) {
        $key = '';

        $elements = ['select', 'where', 'group', 'order', 'limit', 'offset', 'having', 'join'];
        foreach ($elements as $element) {
            $elem = $query->clause($element);
            if ($elem != null && !empty($elem)) {
                $key .= serialize($elem);
            }
        }
        return strtolower($this->table()->getAlias()) . '_' . md5($key);
    }
public function beforeFind($event, SelectQuery $query, $options, $primary) {
        if ($this->isEnabled) {
            $key = $this->cacheKey($query);
            // Set query cache parameter.
            // If cache does not hit and this parameter exist, select results from DB store in cache.
            $query->cache($key, 'find-in-cache');
            $results = Cache::read($key, 'find-in-cache');
            // Stop after events include find event
            if (isset($results) && $results !== false) {
                $query->setResult($results);
                $event->stopPropagation();
                // Disable _decorateResults
                // Because cache is stored after decorate result sets, decorate is executed double.
                $overwrite = true;
                $query->mapReduce(null, null, $overwrite);
                $query->formatResults(null, $overwrite);
                // Disable caching for this query used again
                $query->cache(false);
            }
        }
    }
public function afterSave($event, $entity, $options) {
        if ($this->isEnabled) {
            $alias = strtolower($this->_table->getAlias());
            Cache::clearGroup($alias, 'find-in-cache');
            foreach ($this->_table->associations()->keys() as $assoc) {
                Cache::clearGroup(strtolower($assoc), 'find-in-cache');
            }
        }
    }

    public function afterDelete($event, $entity, $options) {
        if ($this->isEnabled) {
            $alias = strtolower($this->_table->getAlias());
            Cache::clearGroup($alias, 'find-in-cache');
            foreach ($this->_table->associations()->keys() as $assoc) {
                Cache::clearGroup(strtolower($assoc), 'find-in-cache');
            }
        }
    }

With this behavior it should work, but every time a clearGroup being called all the cache groups are invalidated. I tried to debug what is the problem and found that in Cake\Cache\CaheEngine this

protected function _key($key): string
    {
        $this->ensureValidKey($key);

        $prefix = '';
        if ($this->_groupPrefix) {
            $prefix = md5(implode('_', $this->groups()));
        }
        $key = preg_replace('/[\s]+/', '_', $key);

        return $this->_config['prefix'] . $prefix . $key;
    }

function causing the problem. Here you implode all the cacheGroups with there group cache counter and generate an md5 sum from it. So the cache key will look like this:
cacheConfigPreix_generatedMd5FromAllGroups_tableName_selectQueryUnique
But with this behavior every time any cache group counter is being incremented the md5 sum will change therefore all the cache groups are invalidated. My solution was this:

protected function _key(string $key): string
    {
        $this->ensureValidKey($key);

        $prefix = '';
        if ($this->_groupPrefix) {
            $prefix = md5($this->groups()[array_search(substr($key, 0, strpos($key, '_')), $this->_config['groups'])]);
        }
        $key = preg_replace('/[\s]+/', '_', $key);

        return $this->_config['prefix'] . $prefix . $key;
    }

With this fix if groups are defined the keys will have there own group unique in them, not all the groups unique. Now keys look like this:
cacheConfigPreix_generatedMd5FromActualGroup_tableName_selectQueryUnique

With this behavior now cache is working, and after save will only invalidate the actual tables cache and the tables associated wih the actual cache.
I also define cache groups dynamically with this code:

<?php
namespace App\Cache\Engine;

use Cake\Cache\Engine\AnyEngine;
use Cake\ORM\TableRegistry;

class CustomEngine extends AnyEngine
{
    public function init(array $config = []): bool
    {
        
        $groups = [];
        $query = TableRegistry::getTableLocator()->get('Anytable')->getConnection()->execute("show tables")->fetchAll();
        foreach ($query as $t) {
            if (!in_array($t[0], $groups)) {
                $groups[] = $t[0];
            }
        }
        $config['groups'] = array_unique($groups);
            
        return parent::init($config);
        
    }
    
    protected function _key(string $key): string
    {
        $this->ensureValidKey($key);

        $prefix = '';
        if ($this->_groupPrefix) {
            $prefix = md5($this->groups()[array_search(substr($key, 0, strpos($key, '_')), $this->_config['groups'])]);
        }
        $key = preg_replace('/[\s]+/', '_', $key);

        return $this->_config['prefix'] . $prefix . $key;
    }
}

I’m sure that there is a more elegant solution for this, and I’m not sure if this is an actual problem or just something wrong in my logic, but I had this problem since CakePHP 3, and now finally I had the time to debug it, and I don’t understand what is the meaning of group caching with the implemented logic. I hope this could help others who have the same problem as me.

We are currently in the midst of our Cakefest 2023 so most of us are not able to look at this right now.

Quickly scimming through this though it looks like a bug.

Now that I finally fixed the groupCaching, I run in to an other problem. I found that there are few times when I try to clear a group, but the group counter in the cache key wont change. But other counters that should not change is changing. I tried to debug this problem and found, that in Cake\Cache\Engine\MemcachedEngine there is a function groups(). In this function we get all the counters for the groups from the memcached, and generate and array with the groups and the counters together. We use this when we generate the cache keys. But unfortunately memcached in windows will NOT give back the result in the same order as we asked. It might be only windows memcached problem, i only tested this in windows. There is a ksort in the function, but only if the retrieved dataset count is not the same as the groupcount, so when we initalize the group counts.

public function groups(): array
    {
        if (empty($this->_compiledGroupNames)) {
            foreach ($this->_config['groups'] as $group) {
                $this->_compiledGroupNames[] = $this->_config['prefix'] . $group;
            }
        }

        $groups = $this->_Memcached->getMulti($this->_compiledGroupNames) ?: [];
        if (count($groups) !== count($this->_config['groups'])) {
            foreach ($this->_compiledGroupNames as $group) {
                if (!isset($groups[$group])) {
                    $this->_Memcached->set($group, 1, 0);
                    $groups[$group] = 1;
                }
            }
            ksort($groups);
        }

        $result = [];
        $groups = array_values($groups);
        foreach ($this->_config['groups'] as $i => $group) {
            $result[] = $group . $groups[$i];
        }

        return $result;
    }

Example of the error:

$this->_Memcached->getMulti($this->_compiledGroupNames)
[
 (int) 0 => 'data_cache_activedatasets',
 (int) 1 => 'data_cache_activeenvironments',
 (int) 2 => 'data_cache_activepermissions',
 (int) 3 => 'data_cache_activevpns',
 (int) 4 => 'data_cache_applicationenvironmentrels',
 (int) 5 => 'data_cache_applicationexecuterrels',
 (int) 6 => 'data_cache_applicationownerrels',
 (int) 7 => 'data_cache_applications',
 (int) 8 => 'data_cache_companies',
 (int) 9 => 'data_cache_datacategories',
 (int) 10 => 'data_cache_datasets',
 (int) 11 => 'data_cache_departments',
 (int) 12 => 'data_cache_emailtemplates',
 (int) 13 => 'data_cache_employees',
 (int) 14 => 'data_cache_environments',
 (int) 15 => 'data_cache_filters',
 (int) 16 => 'data_cache_jobtitles',
 (int) 17 => 'data_cache_logactions',
 ...
 (int) 86 => 'data_cache_workingdays',
 (int) 87 => 'data_cache_worksteps',
]
$this->_compiledGroupNames
[
 'data_cache_activedatasets' => (int) 1,
 'data_cache_activepermissions' => (int) 1,
 'data_cache_activevpns' => (int) 1,
 'data_cache_applicationownerrels' => (int) 1,
 'data_cache_applications' => (int) 1,
 'data_cache_datacategories' => (int) 1,
 'data_cache_departments' => (int) 1,
 'data_cache_environments' => (int) 3,
 'data_cache_logactions' => (int) 9,
 'data_cache_logapplicationenvironmentrels' => (int) 1,
 'data_cache_logapplicationexecuterrels' => (int) 1,
 'data_cache_logapplicationownerrels' => (int) 1,
 'data_cache_logcompanies' => (int) 1,
 'data_cache_logdatacategories' => (int) 1,
 'data_cache_logdatasets' => (int) 1,
 'data_cache_logdepartments' => (int) 1,
 'data_cache_logemailtemplates' => (int) 1,
 'data_cache_logoutsiders' => (int) 1,
 ...
 'data_cache_workflows' => (int) 5,
 'data_cache_worksteps' => (int) 5,
]

If I move ksort in the function outside of the initalize part, it will work fine. So the fixed function looks like this:

public function groups(): array
    {
        if (empty($this->_compiledGroupNames)) {
            foreach ($this->_config['groups'] as $group) {
                $this->_compiledGroupNames[] = $this->_config['prefix'] . $group;
            }
        }
        $groups = $this->_Memcached->getMulti($this->_compiledGroupNames) ?: [];
        if (count($groups) !== count($this->_config['groups'])) {
            foreach ($this->_compiledGroupNames as $group) {
                if (!isset($groups[$group])) {
                    $this->_Memcached->set($group, 1, 0);
                    $groups[$group] = 1;
                }
            }
        }
        ksort($groups);
        $result = [];
        $groups = array_values($groups);
        foreach ($this->_config['groups'] as $i => $group) {
            $result[] = $group . $groups[$i];
        }

        return $result;
    }

So now after some other fixing my CustomcacheEngine looks like this:

<?php
namespace App\Cache\Engine;

use Cake\Cache\Engine\AnyEngine;
use Cake\Datasource\ConnectionManager;

class CustomEngine extends AnyEngine
{
    public function init(array $config = []): bool
    {
        if ($config != 'debug_kit') {
            $groups = [];
            foreach (ConnectionManager::configured() as $configured) {
                if ($configured != 'debug_kit') {
                    $groups = array_merge($groups, ConnectionManager::get($configured)->getSchemaCollection()->listTables());
                }
            }
            $groups = array_unique($groups);
            sort($groups);
            $config['groups'] = $groups;
        
        }
        return parent::init($config);
        
    }
	
	public function groups(): array
    {
        if (empty($this->_compiledGroupNames)) {
            foreach ($this->_config['groups'] as $group) {
                $this->_compiledGroupNames[] = $this->_config['prefix'] . $group;
            }
        }
        $groups = $this->_Memcached->getMulti($this->_compiledGroupNames) ?: [];
        if (count($groups) !== count($this->_config['groups'])) {
            foreach ($this->_compiledGroupNames as $group) {
                if (!isset($groups[$group])) {
                    $this->_Memcached->set($group, 1, 0);
                    $groups[$group] = 1;
                }
            }
        }
        ksort($groups);
        $result = [];
        $groups = array_values($groups);
        foreach ($this->_config['groups'] as $i => $group) {
            $result[] = $group . $groups[$i];
        }

        return $result;
    }
    
    protected function _key(string $key): string
    {
        $this->ensureValidKey($key);

        $prefix = '';
        if ($this->_groupPrefix) {
            $prefix = md5($this->groups()[array_search(substr($key, 0, strpos($key, '_')), $this->_config['groups'])]);
        }
        $key = preg_replace('/[\s]+/', '_', $key);

        return $this->_config['prefix'] . $prefix . $key;
    }
}

I hope this helps to others having the same strange issues with groupCaching.