Skip to content

/ go Public

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

proposal: runtime/debug: soft memory limit #48409

Open
mknyszek opened this issue Sep 15, 2021 · 5 comments
Open

proposal: runtime/debug: soft memory limit #48409

mknyszek opened this issue Sep 15, 2021 · 5 comments
Labels
GarbageCollector Proposal
Projects
Proposals
Milestone
Proposal

Comments

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Sep 15, 2021 •
edited

Proposal: Soft memory limit

Author: Michael Knyszek

Summary

I propose a new option for tuning the behavior of the Go garbage collector by setting a soft memory limit on the total amount of memory that Go uses.

This option comes in two flavors: a new runtime/debug function called SetMemoryLimit and a GOMEMLIMIT environment variable. In sum, the runtime will try to maintain this memory limit by limiting the size of the heap, and by returning memory to the underlying platform more aggressively. This includes with a mechanism to help mitigate garbage collection death spirals. Finally, by setting GOGC=off, the Go runtime will always grow the heap to the full memory limit.

This new option gives applications better control over their resource economy. It empowers users to:

  • Better utilize the memory that they already have,
  • Confidently decrease their memory limits, knowing Go will respect them,
  • Avoid unsupported forms of garbage collection tuning.

Details

Full design document found here.

Note that, for the time being, this proposal intends to supersede #44309. Frankly, I haven't been able to find a significant use-case for it, as opposed to a soft memory limit overall. If you believe you have a real-world use-case for a memory target where a memory limit with GOGC=off would not solve the same problem, please do not hesitate to post on that issue, contact me on the gophers slack, or via email at mknyszek@golang.org. Please include as much detail as you can.
@mknyszek mknyszek added this to the Go1.18 milestone Sep 15, 2021
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Sep 15, 2021
@gopherbot
Copy link

@gopherbot gopherbot commented Sep 15, 2021

Change https://golang.org/cl/350116 mentions this issue: design: add proposal for a soft memory limit

gopherbot pushed a commit to golang/proposal that referenced this issue Sep 21, 2021
@mknyszek
design: add proposal for a soft memory limit
150687b 
For golang/go#48409.

Change-Id: I4e5d6d117982f51108dca83a8e59b118c2b6f4bf
Reviewed-on: https://go-review.googlesource.com/c/proposal/+/350116
Reviewed-by: Michael Pratt <mpratt@google.com>
@mpx
Copy link
Contributor

@mpx mpx commented Sep 21, 2021

Afaict, the impact of memory limit is visible once the GC is CPU throttled, but not before. Would it be worth exposing the current effective GOGC as well?

@mknyszek
Copy link
Contributor Author

@mknyszek mknyszek commented Sep 21, 2021

@mpx I think that's an interesting idea. If GOGC is not off, then you have a very clear sign of throttling in telemetry. However, if GOGC=off I think it's harder to tell, and it gets blurry once the runtime starts bumping up against the GC CPU utilization limit, i.e. what does effective GOGC mean when the runtime is letting itself exceed the heap goal?

I think that's pretty close. Ideally we would have just one metric that could show, at-a-glance, "are you in the red, and if so, how far?"

@mknyszek mknyszek removed this from the Go1.18 milestone Sep 22, 2021
@mknyszek mknyszek added this to the Proposal milestone Sep 22, 2021
@raulk
Copy link

@raulk raulk commented Sep 27, 2021 •
edited

In case you find this useful as a reference (and possibly to include in "prior art"), the go-watchdog library schedules GC according to a user-defined policy. It can infer limits from the environment/host, container, and it can target a maximum heap size defined by the user. I built this library to deal with #42805, and ever since we integrated it into https://github.com/filecoin-project/lotus, we haven't had a single OOM reported.

@rsc
Copy link
Contributor

@rsc rsc commented Oct 6, 2021

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals Oct 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
GarbageCollector Proposal
Projects
Proposals
Active
Milestone
Proposal
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
@rsc @raulk @mknyszek @mpx @gopherbot