Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs/vfs: unclear description of --vfs-cache-max-age #5793

Open
sevospl opened this issue Nov 6, 2021 · 5 comments
Open

docs/vfs: unclear description of --vfs-cache-max-age #5793

sevospl opened this issue Nov 6, 2021 · 5 comments
Assignees
@sevospl
Labels
doc fix good first issue VFS / mount
Milestone
v1.58

Comments

@sevospl
Copy link

@sevospl sevospl commented Nov 6, 2021

Hi. First of all, kudos to you for the VFS cache - it's really brilliant. I have one request though.

For a little background, as of now I'm using the following settings:

--vfs-cache-mode full \
  --vfs-cache-max-size 500G \
  --vfs-cache-max-age 5000h \
  --vfs-cache-poll-interval 5m \
  --vfs-read-ahead 2G \
  --bwlimit-file 32M \
  --vfs-read-chunk-size=64M \
  --vfs-read-chunk-size-limit=2048M \

I allocate 500 GB to the VFS cache. Right now, because of how vfs-cache-max-age is set, when I run out of space - the oldest files will be removed to free it up. It's already a nice solution, however I'd like to have a bit more flexibility:

1 - Delete cache files that haven't be accessed is X days
2 - When you run out of space => delete the least opened files

The first option would be nice because oldest != less relevant.
There may be very old files that still get accessed and shouldn't be removed.

What do you guys think? I think it would be a nice thing to have.
@ncw
Copy link
Member

@ncw ncw commented Nov 7, 2021

The age used in the vfs-cache-max-age flag is the age of the last access rather than the age of the file.

So I think the VFS already works how you'd like it to!

@sevospl
Copy link
Author

@sevospl sevospl commented Nov 7, 2021

The age used in the vfs-cache-max-age flag is the age of the last access rather than the age of the file.

So I think the VFS already works how you'd like it to!

Sorry but vfs-cache-max-age is confusing then because the option implies it's about the age of files.

My logic here was like this: set the max age as high as possible and rclone will loop over files beginning from those oldest ones to newer and that's what's going to get deleted in order to make some room.

But if it's about the age of files in terms of when they were last opened, then that's perfect!

I think this issue may be closed but maybe you guys want to discuss how docs could be extended to make this a bit clearer.

@ivandeex ivandeex added doc fix and removed question labels Nov 7, 2021
@ivandeex ivandeex added this to the v1.58 milestone Nov 7, 2021
@ivandeex ivandeex changed the title Feature request: More VFS Caching Options docs/vgs: unclear description of --vfs-cache-max-age Nov 7, 2021
@ivandeex
Copy link
Member

@ivandeex ivandeex commented Nov 7, 2021

@albertony @sevospl
Could one of you submit a doc fix?

@ivandeex ivandeex changed the title docs/vgs: unclear description of --vfs-cache-max-age docs/vfs: unclear description of --vfs-cache-max-age Nov 7, 2021
@ncw
Copy link
Member

@ncw ncw commented Nov 8, 2021

The age used in the vfs-cache-max-age flag is the age of the last access rather than the age of the file.
So I think the VFS already works how you'd like it to!

Sorry but vfs-cache-max-age is confusing then because the option implies it's about the age of files.

Yes, it is definitely confusing!

I was using the word "age" as a bit of caching jargon - meaning how long since the file has last been accessed, rather than anything to do with the modification time of the file.

I agree that could be a lot clearer in the docs.

My logic here was like this: set the max age as high as possible and rclone will loop over files beginning from those oldest ones to newer and that's what's going to get deleted in order to make some room.

But if it's about the age of files in terms of when they were last opened, then that's perfect!

I think this issue may be closed but maybe you guys want to discuss how docs could be extended to make this a bit clearer.

I think the only mention of "age" in the docs is

--vfs-cache-max-age duration         Max age of objects in the cache (default 1h0m0s)

We can't change the name of the flag, but we could adjust the help. How about

--vfs-cache-max-age duration         Max time without access of objects in the cache (default 1h0m0s)

?

@sevospl
Copy link
Author

@sevospl sevospl commented Nov 8, 2021 •
edited

Hi! I think it's a good starting point, however I'd like to make it a bit verbose. While you could keep the short description as is, it wouldn't hurt to add a broader explanation because that could be beneficial for some people.

E.g.:

--vfs-cache-age duration

When --vfs-cache-max-size is in use, rclone will attempt to flush the unaccessed files in a long time from cache first. Thus the higher this duration value is - the higher the likehood is that rclone will start from the least relevant files that don't need to stay in cache which makes the cache flushing strategy more efficient and more relevant files are likely to remain cached.

I'm not a native speaker so feel free to rewrite this :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sevospl sevospl

Labels
doc fix good first issue VFS / mount
Projects
None yet
Milestone
v1.58
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
@ncw @ivandeex @sevospl