Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: invalid frame pointer #32

Merged
merged 3 commits into from
Nov 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

Goref is a Go heap object reference analysis tool based on delve.

It can display the space and object count distribution of Go memory references, which is helpful for efficiently locating memory leak issues or viewing persistent heap objects to optimize GC overhead.
It can display the space and object count distribution of Go memory references, which is helpful for efficiently locating memory leak issues or viewing persistent heap objects to optimize the garbage collector (GC) overhead.

## Installation

Expand All @@ -25,15 +25,15 @@ $ go tool pprof -http=:5079 ./grf.out

The opened HTML page displays the reference distribution of the heap memory. You can choose to view the "inuse space" or "inuse objects".

For example, the following picture is a Heap Profile of pprof. It can be seen that the objects are mainly allocated by `FastRead` function, which is Kitex's deserialization code. This flame graph is not very helpful for troubleshooting because memory allocation for decoding and constructing data is normal.
For example, the heap profile sampled from a [testing program](https://github.com/cloudwego/goref/blob/main/testdata/mockleak/main.go) is shown below, which reflects the call stack distribution of object creation.

![image](https://github.com/user-attachments/assets/799c1b9a-fcf0-4b35-ab15-03a2bf3a919e)
![img_v3_02gq_63631612-6f2d-40ce-8f98-a4e52682ef7g](https://github.com/user-attachments/assets/9fb6bded-3f68-4b73-972d-a273c45b7680)

However, by using the goref tool, the following results can be seen: `mockCache` holding onto RPC's Response causing memory not to be released, making the problem clear at a glance.

![image](https://github.com/user-attachments/assets/1c3d5d29-953d-4364-84b9-69ae35f51152)
By using the goref tool, you can see the memory reference distribution of heap objects reachable by GC, thereby quickly pinpointing the actual code locations holding references.

![img_v3_02gq_53dc2cff-203a-4c06-9678-aae49da4754g](https://github.com/user-attachments/assets/7a6b5a83-e3cd-415f-a5c0-c88d6493e45b)

![img_v3_02gq_54551396-a4ae-42b8-996f-1b1699d381dg](https://github.com/user-attachments/assets/2466c26a-eb78-4be9-af48-7a25e851982a)

It also supports analyzing core files, e.g.

Expand Down
6 changes: 5 additions & 1 deletion pkg/proc/heap.go
Original file line number Diff line number Diff line change
Expand Up @@ -555,7 +555,7 @@ func (s *HeapScope) parseSegment(name string, md *region) *segment {
// which is extremely complex to handle and is not conducive to the maintenance of the project.
// Therefore, Goref adopts a conservative scanning scheme.
// NOTE: This may lead to scanning an additional portion of memory.
func (s *HeapScope) stackPtrMask(frames []proc.Stackframe) []*framePointerMask {
func (s *HeapScope) stackPtrMask(start, end Address, frames []proc.Stackframe) []*framePointerMask {
var frPtrMasks []*framePointerMask
for i := range frames {
pc := frames[i].Regs.PC()
Expand All @@ -565,6 +565,10 @@ func (s *HeapScope) stackPtrMask(frames []proc.Stackframe) []*framePointerMask {
}
sp := Address(frames[i].Regs.SP())
fp := Address(frames[i].Regs.FrameBase)
if fp <= sp || fp > end || sp < start {
// invalid frame pointer
continue
}
ptrMask := make([]uint64, CeilDivide(fp.Sub(sp)/8, 64))
for i := range ptrMask {
ptrMask[i] = ^uint64(0)
Expand Down
2 changes: 1 addition & 1 deletion pkg/proc/reference.go
Original file line number Diff line number Diff line change
Expand Up @@ -507,7 +507,7 @@ func ObjectReference(t *proc.Target, filename string) error {
threadID = gr.Thread.ThreadID()
}
sf, _ := proc.GoroutineStacktrace(t, gr, 1024, 0)
s.g.init(Address(lo), Address(hi), s.stackPtrMask(sf))
s.g.init(Address(lo), Address(hi), s.stackPtrMask(Address(lo), Address(hi), sf))
if len(sf) > 0 {
for i := range sf {
ms := myEvalScope{EvalScope: *proc.FrameToScope(t, t.Memory(), gr, threadID, sf[i:]...)}
Expand Down
54 changes: 54 additions & 0 deletions testdata/mockleak/creator/creator.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
package creator

type InnerMessage struct {
msgs []string
}

type MyChan struct {
cchan chan *InnerMessage
}

type SubRequest struct {
E map[string]string
F map[int64]*MyChan
}

type Request struct {
A *int64
B *string
C []string
D *SubRequest
X []*SubRequest
}

func Create() interface{} {
buf := make([]byte, 1024)
str := string(buf)
return &Request{
A: new(int64),
B: &str,
C: []string{string(buf), string(buf), string(buf)},
D: &SubRequest{
E: map[string]string{
string(buf): string(buf),
},
F: map[int64]*MyChan{
123456: {
cchan: make(chan *InnerMessage, 100),
},
},
},
X: []*SubRequest{
{
E: map[string]string{
string(buf): string(buf),
},
F: map[int64]*MyChan{
123456: {
cchan: make(chan *InnerMessage, 100),
},
},
},
},
}
}
12 changes: 12 additions & 0 deletions testdata/mockleak/holder/holder.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package holder

func ReceiveChan() chan interface{} {
ch := make(chan interface{}, 100)
go func() {
var cached []interface{}
for v := range ch {
cached = append(cached, v)
}
}()
return ch
}
22 changes: 22 additions & 0 deletions testdata/mockleak/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package main

import (
"log"
"net/http"
_ "net/http/pprof"
"time"

"github.com/cloudwego/goref/testdata/mockleak/creator"
"github.com/cloudwego/goref/testdata/mockleak/holder"
)

func main() {
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()
receiver := holder.ReceiveChan()
for i := 0; i < 100000; i++ {
receiver <- creator.Create()
}
time.Sleep(1 * time.Minute)
}
Loading