Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The settings related to memory size don't work #630

Closed
kamikyo opened this issue Sep 3, 2024 · 5 comments
Closed

The settings related to memory size don't work #630

kamikyo opened this issue Sep 3, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@kamikyo
Copy link

kamikyo commented Sep 3, 2024 •
edited
Loading

Describe the bug

I continuously write data to Garnet, primarily with three keys: two sorted sets and one string. Each time, I write 5000 entries, approximately 1 to 2 MB. Garnet's memory keeps increasing over time. As long as I keep writing data, the memory growth will not stop.

Steps to reproduce the bug

garnet server's code:

public static void Main(string[] args)
{
    try
    {
        var factory = LoggerFactory.Create(builder =>
        {
            builder.AddConsole(options =>
            {
                options.LogToStandardErrorThreshold = LogLevel.Trace;
            });
        });
        using var server = new GarnetServer(args, factory);
        server.Start();
        Thread.Sleep(Timeout.Infinite);
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Unable to initialize server due to exception: {ex.Message}");
    }
}

the parameters I specified are:
--port 7010 --index "256m" --index-max-size "512m" --obj-index-max-size "512m" --memory "1g" --recover true --compaction-type "Scan" --checkpointdir "data" --storage-tier true

Expected behavior

I expect the memory to stabilize after reaching a certain limit.

Screenshots

No response

Release version

v1.0.15

IDE

Visual studio 2022 community

OS version

windows 10

Additional context

After each execution of the save command, memory usage increases by 200-300 MB.
@badrishc
Copy link
Contributor

badrishc commented Sep 3, 2024

Can you provide a sample client app as well? Try compaction type of Lookup.

If you are adding new entries in the sorted set, the total memory is expected to grow.

Also, it might be GC uncollected memory - try issuing the GC collection (FORCEGC) using the remote client command line.

@kamikyo
Copy link
Author

kamikyo commented Sep 4, 2024

@badrishc
The code for the garnet server is as described above. It is a console application(GarnetSample.csproj), and after publishing, it generates an executable file named GarnetSample.exe.
Server's start command:
GarnetSample.exe --port 7010 --index "256m" --index-max-size "512m" --obj-index-max-size "512m" --memory "1g" --recover true --compaction-type "Scan" --checkpointdir "data" --storage-tier true

Client project is DataCreator.csproj.It writes 5,000 records to the garnet server every 3 seconds.
The code repository is located at: https://github.com/kamikyo/garnetdemo

@badrishc
Copy link
Contributor

badrishc commented Sep 4, 2024

Does the problem persist if you use CompactionType of Lookup?

@darrenge darrenge added the bug Something isn't working label Sep 5, 2024
@badrishc
Copy link
Contributor

Ping - can you (1) try with the latest Garnet version; (2) definitely switch to CompactionType of Lookup. We will most likely deprecate the Scan compaction type soon.

@yrajas
Copy link
Contributor

yrajas commented Oct 7, 2024

This PR #697 modifies some of the memory controlling switches to simplify memory utilization management. Please try with the new switches --obj-log-memory and --obj-heap-memory. Documentation available here https://microsoft.github.io/garnet/docs/getting-started/memory#object-store.

Also, to control memory management behavior of GC try working with the GC settings too like https://learn.microsoft.com/en-us/dotnet/core/runtime-config/garbage-collector#dynamic-adaptation-to-application-sizes-datas, https://learn.microsoft.com/en-us/dotnet/core/runtime-config/garbage-collector#conserve-memory.

Closing this issue now. Please open a new one if you encounter any Garnet issues.

@yrajas yrajas closed this as completed Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@darrenge @kamikyo @badrishc @yrajas