目前使用的程式邏輯會在瞬間創造出大量的goroutine,但發現goroutine結束任務後,基本的空間無法被回收.
使用pprof監測(最下方有範例程式):
查看記憶體
(pprof) top
Showing nodes accounting for 16035.46kB, 100% of 16035.46kB total
Showing top 10 nodes out of 27
flat flat% sum% cum cum%
9731.56kB 60.69% 60.69% 9731.56kB 60.69% runtime.malg
4734.69kB 29.53% 90.21% 4734.69kB 29.53% golang.org/x/net/webdav.(*memFile).Write
(pprof) list runtime.malg
Total: 15.66MB
ROUTINE ======================== runtime.malg in C:\go\src\runtime\proc.go
9.50MB 9.50MB (flat, cum) 60.69% of Total
. . 3983: execLock.unlock()
. . 3984:}
. . 3985:
. . 3986:// Allocate a new g, with a stack big enough for stacksize bytes.
. . 3987:func malg(stacksize int32) *g {
9.50MB 9.50MB 3988: newg := new(g)
雖然記憶體在全部goroutine跑完有下降一點,但無法恢復到原來未創建前的狀態,而這一段newg看起來是用於創建goroutine,我以為goroutine只要有正常結束應該會自動回收掉?
找到相關的問題
1.go runtime.malg导致的内存泄漏
https://blog.csdn.net/wuyuhao13579/article/details/109079570
去搜寻了大量资料之后,发现go的官网早就有这个issue(官方issue),大佬们知道,只是不好解决
(...這篇好深 看不懂解法)
go issue
https://github.com/golang/go/issues/34457
2.goroutine栈的申请与释放
https://blog.haohtml.com/archives/30403
当一个g运行结束的时候,可能会释放stack(只是有可能),原因在於可能被復用?
想問一下各位如果過程中有可能創造這麼多的goroutine,基礎需要的空間就不會回收嗎?沒有方法可以回收掉嗎?
================
簡易範例:
package main
import (
"fmt"
"github.com/gin-contrib/pprof"
"github.com/gin-gonic/gin"
)
func demoFunc(i int) {
fmt.Println("Hello World!,I am done.", i)
}
func main() {
gin.SetMode(gin.ReleaseMode)
engine := gin.New()
pprof.Register(engine) // pprof monitor
endpoint := fmt.Sprintf(":%d", 8888)
go func() { //分析用
err := engine.Run(endpoint)
if err != nil {
fmt.Println(err)
}
}()
runTimes := 1048575 //給他一個大數目創建go
for i := 0; i < runTimes; i++ {
go demoFunc(i)
}
fmt.Printf("end call")
select {} //避免退出
}
分析過程:
直接開啟網頁
http://localhost:8888/debug/pprof/
Types of profiles available:
Count Profile
32 allocs
0 block
0 cmdline
776924 goroutine
32 heap
// 等待一段時間再問一次
Types of profiles available:
Count Profile
33 allocs
0 block
0 cmdline
5 goroutine
33 heap
再下指令查詢記憶體
>go tool pprof http://localhost:8888/debug/pprof/heap?debug
Fetching profile over HTTP from http://localhost:8888/debug/pprof/heap?debug=
\pprof.alloc_objects.alloc_space.inuse_objects.inuse_space.248.pb.gz
Type: inuse_space
Time: Oct 28, 2021 at 11:05am (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 569.83MB, 99.91% of 570.34MB total
Dropped 8 nodes (cum <= 2.85MB)
Showing top 10 nodes out of 24
flat flat% sum% cum cum%
369.64MB 64.81% 64.81% 369.64MB 64.81% runtime.malg
108.02MB 18.94% 83.75% 108.02MB 18.94% fmt.glob..func1
52.50MB 9.21% 92.96% 52.50MB 9.21% internal/poll.runtime_Semacquire
25.50MB 4.47% 97.43% 25.50MB 4.47% fmt.(*buffer).write (inline)
8.67MB 1.52% 98.95% 8.67MB 1.52% runtime.allgadd
5.50MB 0.96% 99.91% 191.53MB 33.58% main.demoFunc
0 0% 99.91% 25.50MB 4.47% fmt.(*fmt).fmtInteger
0 0% 99.91% 25.50MB 4.47% fmt.(*fmt).pad
0 0% 99.91% 25.50MB 4.47% fmt.(*pp).doPrintln
0 0% 99.91% 25.50MB 4.47% fmt.(*pp).fmtInteger
-----------
Dropped 8 nodes (cum <= 1.96MB)
flat flat% sum% cum cum%
383.14MB 97.66% 97.66% 383.14MB 97.66% runtime.malg
8.67MB 2.21% 99.87% 8.67MB 2.21% runtime.allgadd
0 0% 99.87% 391.81MB 99.87% runtime.mstart
0 0% 99.87% 391.81MB 99.87% runtime.newproc.func1
0 0% 99.87% 391.81MB 99.87% runtime.newproc1
0 0% 99.87% 391.81MB 99.87% runtime.systemstac
go version &env
go version go1.17.6 darwin/amd64
也有在windows64系統上試過
go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/**/Library/Caches/go-build"
GOENV="/Users/**/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/**/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/**/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.6"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/**/go/src/simple_go_demo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mj/nl_h62sx3wd3bj4fgqhhngfw0000gn/T/go-build3714213462=/tmp/go-build -gno-record-gcc-switches -fno-common"
像Go, Java, node, .net....等等,除非編譯成原生的執行檔,否則應該都是在一個虛擬機環境底下跑你的程式,釋放記憶體就要看他們的虛擬機怎麼實做垃圾收集(GC, Garbage Collection)。
參考:https://medium.com/@openmohan/go-scheduling-and-garbage-collection-91b5144bc26b
依照文章建議,也許可以考慮在適當時機呼叫runtime.GC()
。我沒在使用Go,但是多少接觸過不同語言的垃圾收集相關的議題,我想Go在這方面的機制也是需要瞭解的。
試過呼叫runtime.GC(),仍沒有效果.QQ
你貼的文章,裡面的連結有提到作者另一篇分析:
https://blog.haohtml.com/archives/23437
基本上提到一些不釋放的理由,另外有一些規則...都是google的作品,所以一些哲學很像,chrome也是用資源換取性能。