My first solution was a pure lua script, but as this did not work due to the redis cluster issue, I went with a more straight forward "iterating over the keys in php"-solution. But even there I ran into some issues with the redis cluster (namely accessing a key on the wrong node).

Keep in mind that this solution might also be bad in regards to performance when it comes to having millions of keys/tags in redis, as my thought was, to make it work first and improve upon later

I added a few comments to hopefully make it more clear what I was trying to do.

public function prune(): bool
{
    if ($this->redis instanceof \Predis\ClientInterface) {
        $prefix = $this->redis->getOptions()->prefix ? $this->redis->getOptions()->prefix->getPrefix() : '';
    } elseif (\is_array($prefix = $this->redis->getOption(\Redis::OPT_PREFIX) ?? '')) {
        $prefix = current($prefix);
    }

    // get all SET entries that are symfony "tag" related
    // note: it is not possible to use \0 in lua (which is used as tag prefix in symfony), so as of right now every key that has "tags" in it will be fetched
    $getTagsLua = <<<'EOLUA'
    redis.replicate_commands()

    local cursor = ARGV[1]
    local prefix = ARGV[2]
    local tagPrefix = string.gsub(KEYS[1], prefix, "")

    return redis.call('SCAN', cursor, 'COUNT', 5000, 'MATCH', '*' .. tagPrefix .. '*', 'TYPE', 'set')
EOLUA;

    // lua for fetching all entries from a SET
    $getSetKeysLua = <<<'EOLUA'
    redis.replicate_commands()

    local cursor = ARGV[1]

    return redis.call('SSCAN', KEYS[1], cursor, 'COUNT', 5000)
EOLUA;

    $cursor = '0';
    do {
        $results = $this->pipeline(function () use ($getTagsLua, $cursor, $prefix) {
            yield 'eval' => [$getTagsLua, ['tags', $cursor, $prefix], 1];
        });

        foreach ($results as $values) {
            [$cursor, $ids] = $values;

            $setCursor = '0';
            $hasExistingKeys = false;
            // iterate over each SET key
            foreach ($ids as $id) {
                $id = substr($id, strlen($prefix));
                do {
                    $setResults = $this->pipeline((function () use ($getSetKeysLua, $id, $setCursor) {
                        yield 'eval' => [$getSetKeysLua, [$id, $setCursor], 1];
                    }));

                    foreach ($setResults as [$setCursor, $setIds]) {
                        //noop
                    }

                    if (!empty($setIds)) {
                        $existResult = $this->pipeline(function () use ($setIds) {
                            yield 'exists' => $setIds;
                        });

                        $exists = 0;
                        foreach ($existResult as $exists) {
                            // noop
                        }
                        $hasExistingKeys = $hasExistingKeys || $exists > 0;
                    }
                } while ('0' !== $setCursor);

                // if any of the members in the SET still exists as a key in redis, do not delete the SET
                if (!$hasExistingKeys) {
                    $results = $this->pipeline(function () use ($id) {
                        yield 'del' => [$id];
                    });
                    foreach ($results as $result) {
                        //noop
                    }
                }
            }
        }
    } while ('0' !== $cursor);

    return true;
}

Side note: This code currently does not work with Predis, as the parameter position for the eval function needs to be adjusted accordingly

Uh oh!

Implement PrunableInterface on RedisTagAwareAdapter #47835

Description

Description

Example

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions