{"id":1931,"date":"2022-09-28T10:05:53","date_gmt":"2022-09-28T10:05:53","guid":{"rendered":"https:\/\/www.ipcpu.com\/?p=1931"},"modified":"2022-09-28T10:15:42","modified_gmt":"2022-09-28T10:15:42","slug":"k8s-docker-memory-monitor","status":"publish","type":"post","link":"https:\/\/c.ipcpu.com\/2022\/09\/k8s-docker-memory-monitor\/","title":{"rendered":"K8S\/Docker\u4e2d\u5bf9\u4e8e\u5bb9\u5668\u5185\u5b58\u7684\u76d1\u63a7"},"content":{"rendered":"<div id=\"MathJax_Message\" style=\"display: none;\"><\/div>\n<p><img decoding=\"async\" src=\"https:\/\/www.ipcpu.com\/wp-content\/uploads\/2022\/09\/1deb8a732186b0677a4b41f9a6619fea_7a9c27d9-393c-4fd7-b8d4-27c4a4045383.webp\" \/><\/p>\n<h2 id=\"wiz-toc-0-1442681499\">\u4e00\u3001\u6982\u8ff0<\/h2>\n<p>\u5728\u4f7f\u7528Docker\u6216\u8005Kubernetes\u65f6\uff0c\u6211\u4eec\u7ecf\u5e38\u9700\u8981\u76d1\u63a7\u5bb9\u5668\u6216\u8005Pod\u7684\u5185\u5b58\uff0c\u540c\u65f6\u6211\u4eec\u4e5f\u7ecf\u5e38\u6536\u5230\u53cd\u9988\u5185\u5b58\u4e0d\u51c6\u786e\u7684\u60c5\u51b5\uff0c\u8fd9\u4e0d\u4ec5\u662f\u56e0\u4e3a\u5b58\u5728Buffer\u3001Cache\u7684\u5f71\u54cd\uff0c\u4e0d\u540c\u7684\u7b97\u6cd5\u6307\u6807\u4e5f\u4f1a\u5f97\u51fa\u4e0d\u540c\u7684\u7ed3\u679c\u3002<br \/>\n\u63a5\u4e0b\u6765\u6211\u4eec\u5148\u56de\u987e\u4e0b\u6211\u4eec\u6700\u53e4\u8001\u7684\u8ba1\u7b97\u65b9\u6cd5\uff0c\u7136\u540e\u5206\u522b\u53d6\u5206\u6790docker stats \u548c kubectl top \u4e2d\u7684\u5185\u5b58\u8ba1\u7b97\u65b9\u6cd5\u3002<\/p>\n<h2 id=\"wiz-toc-1-1051446464\">\u4e8c\u3001\u6700\u539f\u59cb\u7684\u8ba1\u7b97\u5185\u5b58\u65b9\u6cd5<\/h2>\n<p>\u6211\u7ffb\u9605\u4e86\u4ee5\u524d\u7684\u4e00\u4e9b\u5de5\u4f5c\u8d44\u6599\uff0c\u53d1\u73b0\u4eceDocker 1.5 \u5f00\u59cb\u624d\u63d0\u4f9b\u4e86Docker stats\u63a5\u53e3\uff0c\u8fd9\u4e2a\u63a5\u53e3\u6709API\u548c\u547d\u4ee4\u884c\u4e24\u79cd\u65b9\u5f0f\uff0c\u5e76\u4e14API\u662fstream\u65b9\u5f0f\uff0c\u4f1a\u4e0d\u95f4\u65ad\u7684\u53d1\u9001\u6570\u636e\uff0c\u5f53\u65f6\u6211\u4eec\u4e5f\u4e0d\u6e05\u695a\u5982\u4f55\u548c\u8fd9\u6837\u7684API\u63a5\u53e3\u5bf9\u63a5\uff0c\u6240\u4ee5\u6211\u4eec\u9009\u62e9\u4e86\u76f4\u63a5\u5bf9cgroup\u6587\u4ef6\u4e0b\u624b\u3002<br \/>\n\u6211\u4eec\u8bfb\u53d6\u4e86\u4ee5\u4e0b\u4e24\u4e2a\u6587\u4ef6\uff1a<\/p>\n<div>\n<pre><code># \u5bb9\u5668\u5f53\u524d\u5df2\u4f7f\u7528\u7684\u5185\u5b58Bytes\n\/cgroup\/memory\/docker\/$DOCKERID\/memory.usage_in_bytes\n# \u5bb9\u5668\u6700\u5927\u9650\u5236\u5185\u5b58Bytes\n\/cgroup\/memory\/docker\/$DOCKERID\/memory.limit_in_bytes<\/code><\/pre>\n<\/div>\n<p>\u8fd9\u4e24\u4e2a\u6570\u636e\u76f8\u9664\u5c31\u5f97\u5230\u4e86\u5185\u5b58\u4f7f\u7528\u7387\uff0c\u5f53\u65f6\u5e76\u6ca1\u6709\u8003\u8651Buffer\u3001Cache\u7b49\u6570\u636e\uff0c\u4e5f\u6ca1\u6709\u8be6\u7ec6\u6df1\u7a76\/cgroup\/memory\/docker\/$DOCKERID\/memory.stat\u6587\u4ef6\u5185\u5bb9\uff0c\u6240\u4ee5\u4e0edocker stats\u7565\u6709\u504f\u5dee\u3002<\/p>\n<p>\u63a5\u4e0b\u6765\u6211\u4eec\u770b\u4e0bDocker stats \u4e2d\u7684\u5b9e\u73b0\u65b9\u6cd5<\/p>\n<h2 id=\"wiz-toc-2-1468301695\">\u4e09\u3001Docker stats\u4e2d\u7684\u5185\u5b58\u8ba1\u7b97\u65b9\u6cd5<\/h2>\n<p>\u7531\u4e8e\u4f4e\u7248\u672c\u7684docker\u6e90\u7801\u5df2\u7ecf\u770b\u4e0d\u4e86\u4e86\uff0c\u5728Docker V19\u548c\u4e4b\u524d\u7684\u7248\u672c\u4e2d\uff0c<\/p>\n<div>\n<pre><code>mem.Usage - mem.Stats[\"cache\"]<\/code><\/pre>\n<\/div>\n<p>\u770b\u8d77\u6765\u4e0e\u6211\u4eec\u4e0a\u9762\u5b9e\u73b0\u7684\u65b9\u5f0f\u7c7b\u4f3c\uff0c\u51cf\u6389\u4e86Cache\u90e8\u5206\uff0c\u5177\u4f53\u6e90\u7801\u5982\u4e0b\uff1a<\/p>\n<div>\n<pre><code>\/\/ calculateMemUsageUnixNoCache calculate memory usage of the container.\n\/\/ Page cache is intentionally excluded to avoid misinterpretation of the output.\nfunc calculateMemUsageUnixNoCache(mem types.MemoryStats) float64 {\n    return float64(mem.Usage - mem.Stats[\"cache\"])\n}<\/code><\/pre>\n<\/div>\n<p>\u4eceDocker V20\u7248\u672c\u5f00\u59cb\u8fd9\u90e8\u5206\u6709\u4fee\u6539\uff0c\u8ba1\u7b97\u65b9\u6cd5\u4e3a<\/p>\n<div>\n<pre><code>mem.Usage - inactive_file<\/code><\/pre>\n<\/div>\n<p>\u8be6\u7ec6\u4ee3\u7801\u5982\u4e0b<\/p>\n<div>\n<pre><code>\/\/ calculateMemUsageUnixNoCache calculate memory usage of the container.\n\/\/ Cache is intentionally excluded to avoid misinterpretation of the output.\n\/\/\n\/\/ On cgroup v1 host, the result is `mem.Usage - mem.Stats[\"total_inactive_file\"]` .\n\/\/ On cgroup v2 host, the result is `mem.Usage - mem.Stats[\"inactive_file\"] `.\n\/\/\n\/\/ This definition is consistent with cadvisor and containerd\/CRI.\n\/\/ * https:\/\/github.com\/google\/cadvisor\/commit\/307d1b1cb320fef66fab02db749f07a459245451\n\/\/ * https:\/\/github.com\/containerd\/cri\/commit\/6b8846cdf8b8c98c1d965313d66bc8489166059a\n\/\/\n\/\/ On Docker 19.03 and older, the result was `mem.Usage - mem.Stats[\"cache\"]`.\n\/\/ See https:\/\/github.com\/moby\/moby\/issues\/40727 for the background.\nfunc calculateMemUsageUnixNoCache(mem types.MemoryStats) float64 {\n    \/\/ cgroup v1\n    if v, isCgroup1 := mem.Stats[\"total_inactive_file\"]; isCgroup1 &amp;&amp; v &lt; mem.Usage {\n        return float64(mem.Usage - v)\n    }\n    \/\/ cgroup v2\n    if v := mem.Stats[\"inactive_file\"]; v &lt; mem.Usage {\n        return float64(mem.Usage - v)\n    }\n    return float64(mem.Usage)\n}<\/code><\/pre>\n<\/div>\n<h2 id=\"wiz-toc-3-1106454420\">\u56db\u3001Kubernetes\/cadvisor\u4e2d\u7684\u5185\u5b58\u8ba1\u7b97\u65b9\u6cd5<\/h2>\n<p>kubectl top \u7684\u5185\u5b58\u6570\u636e\u7528working_set\u53c2\u6570\uff0c\u5e95\u5c42\u6570\u636e\u662f\u4ecekubelet\u4e2d\u5185\u7f6e\u7684cadvisor\u6765\u8bfb\u53d6\u7684\uff0c\u4f46\u662fcadvisor\u7684\u65e7\u4ee3\u7801\u6211\u4eec\u4e5f\u65e0\u8ff9\u53ef\u5bfb\u4e86\uff0c\u6211\u4eec\u627e\u4e86v0.30.0 9 Mar 2018 \uff0c\u8ba1\u7b97\u65b9\u5f0f\u4e3a\uff1a<\/p>\n<div>\n<pre><code>    workingSet := ret.Memory.Usage\n    if v, ok := s.MemoryStats.Stats[\"total_inactive_file\"]; ok {\n        if workingSet &lt; v {\n            workingSet = 0\n        } else {\n            workingSet -= v\n        }\n    }\n    ret.Memory.WorkingSet = workingSet<\/code><\/pre>\n<\/div>\n<p>\u7b80\u5355\u7684\u8bf4\uff0c\u5c31\u662f\uff1a<\/p>\n<div>\n<pre><code>Memory.Usage - MemoryStats.Stats[\"total_inactive_file\"]<\/code><\/pre>\n<\/div>\n<p>\u8fd9\u4e2a\u65b9\u6cd5\u548cDocker V20\u4ee5\u540e\u662f\u4e00\u81f4\u7684\uff0c\u4e5f\u5c31\u662f\u8bf4\u4e00\u76f4\u5230Docker 20.10.0\u65f6\u95f4\u662f2020-12-08\uff0cdocker stats \u548c kubectl top \u5185\u5b58\u6570\u636e\u624d\u8fbe\u6210\u4e00\u81f4\uff0c\u4e4b\u524d\u90fd\u662f\u4e0d\u7edf\u4e00\u7684\u3002<\/p>\n<h2 id=\"wiz-toc-4-225491530\">\u4e94\u3001\u7edf\u4e00\u7684\u5185\u5b58\u8ba1\u7b97\u65b9\u6cd5<\/h2>\n<p>\u6240\u4ee5\uff0cDocker 20.10.0(2020-12-08) \u4e4b\u540e\uff0cdocker stats \u548c kubectl top\u90fd\u4f1a\u4f7f\u7528\u4e0b\u9762\u65b9\u6cd5\u8ba1\u7b97\u5185\u5b58\u4f7f\u7528\uff1a<\/p>\n<div>\n<pre><code>memory_working_set  = Memory.Usage - Memeory.inactive_file<\/code><\/pre>\n<\/div>\n<h2 id=\"wiz-toc-5-666041831\">\u53c2\u8003\u8d44\u6599<\/h2>\n<p>https:\/\/qingwave.github.io\/container-memory\/<br \/>\nhttps:\/\/segmentfault.com\/a\/1190000021493607<br \/>\nhttps:\/\/blog.csdn.net\/Ivan_Wz\/article\/details\/119457692<br \/>\nhttps:\/\/kubesphere.com.cn\/forum\/d\/1517-docker-stats-mem-top-res<br \/>\nhttps:\/\/github.com\/docker\/cli\/blob\/v19.03.15\/cli\/command\/container\/stats_helpers.go<\/p>\n<p>\u8f6c\u8f7d\u8bf7\u6ce8\u660e\uff1a<a href=\"https:\/\/c.ipcpu.com\">IPCPU-\u7f51\u7edc\u4e4b\u8def<\/a> &raquo; <a href=\"https:\/\/c.ipcpu.com\/2022\/09\/k8s-docker-memory-monitor\/\">K8S\/Docker\u4e2d\u5bf9\u4e8e\u5bb9\u5668\u5185\u5b58\u7684\u76d1\u63a7<\/a><\/p>","protected":false},"excerpt":{"rendered":"<p>\u4e00\u3001\u6982\u8ff0 \u5728\u4f7f\u7528Docker\u6216\u8005Kubernetes\u65f6\uff0c\u6211\u4eec\u7ecf\u5e38\u9700\u8981\u76d1\u63a7\u5bb9\u5668\u6216\u8005Pod\u7684\u5185\u5b58\uff0c\u540c\u65f6\u6211\u4eec\u4e5f\u7ecf\u5e38\u6536\u5230\u53cd\u9988\u5185\u5b58\u4e0d\u51c6\u786e\u7684\u60c5\u51b5\uff0c\u8fd9\u4e0d\u4ec5\u662f\u56e0\u4e3a\u5b58\u5728Buffer\u3001Cache\u7684\u5f71\u54cd\uff0c\u4e0d\u540c\u7684\u7b97\u6cd5\u6307\u6807\u4e5f\u4f1a\u5f97\u51fa\u4e0d\u540c\u7684\u7ed3\u679c\u3002 \u63a5\u4e0b\u6765\u6211\u4eec\u5148\u56de\u987e\u4e0b\u6211\u4eec\u6700\u53e4\u8001\u7684\u8ba1\u7b97\u65b9\u6cd5\uff0c\u7136\u540e\u5206\u522b\u53d6\u5206\u6790docker stats \u548c kubectl top \u4e2d\u7684\u5185\u5b58\u8ba1\u7b97\u65b9\u6cd5\u3002 \u4e8c\u3001\u6700\u539f\u59cb\u7684\u8ba1\u7b97\u5185\u5b58\u65b9\u6cd5 \u6211\u7ffb\u9605\u4e86\u4ee5\u524d\u7684\u4e00\u4e9b\u5de5\u4f5c\u8d44\u6599\uff0c\u53d1\u73b0\u4eceDocker 1.5 \u5f00\u59cb\u624d\u63d0\u4f9b\u4e86Docker stats\u63a5\u53e3\uff0c\u8fd9\u4e2a\u63a5\u53e3\u6709API\u548c\u547d\u4ee4\u884c\u4e24\u79cd\u65b9\u5f0f\uff0c\u5e76\u4e14API\u662fstream\u65b9\u5f0f\uff0c\u4f1a\u4e0d\u95f4\u65ad\u7684\u53d1\u9001\u6570\u636e\uff0c\u5f53\u65f6\u6211\u4eec\u4e5f\u4e0d\u6e05\u695a\u5982\u4f55\u548c\u8fd9\u6837\u7684API\u63a5\u53e3\u5bf9\u63a5\uff0c\u6240\u4ee5\u6211\u4eec\u9009\u62e9\u4e86\u76f4\u63a5\u5bf9cgroup\u6587\u4ef6\u4e0b\u624b\u3002 \u6211\u4eec\u8bfb\u53d6\u4e86\u4ee5\u4e0b\u4e24\u4e2a\u6587\u4ef6\uff1a # \u5bb9\u5668\u5f53\u524d\u5df2\u4f7f\u7528\u7684\u5185\u5b58Bytes \/cgroup\/memory\/docker\/$DOCKERID\/memory.usage_in_bytes # \u5bb9\u5668\u6700\u5927\u9650\u5236\u5185\u5b58Bytes \/cgroup\/memory\/docker\/$DOCKERID\/memory.limit_in_bytes \u8fd9\u4e24\u4e2a\u6570\u636e\u76f8\u9664\u5c31\u5f97\u5230\u4e86\u5185\u5b58\u4f7f\u7528\u7387\uff0c\u5f53\u65f6\u5e76\u6ca1\u6709\u8003\u8651Buffer\u3001Cache\u7b49\u6570\u636e\uff0c\u4e5f\u6ca1\u6709\u8be6\u7ec6\u6df1\u7a76\/cgroup\/memory\/docker\/$DOCKERID\/memory.stat\u6587\u4ef6\u5185\u5bb9\uff0c\u6240\u4ee5\u4e0edocker stats\u7565\u6709\u504f\u5dee\u3002 \u63a5\u4e0b\u6765\u6211\u4eec\u770b\u4e0bDocker stats \u4e2d\u7684\u5b9e\u73b0\u65b9\u6cd5 \u4e09\u3001Docker stats\u4e2d\u7684\u5185\u5b58\u8ba1\u7b97\u65b9\u6cd5 \u7531\u4e8e\u4f4e\u7248\u672c\u7684docker\u6e90\u7801\u5df2\u7ecf\u770b\u4e0d\u4e86\u4e86\uff0c\u5728Docker V19\u548c\u4e4b\u524d\u7684\u7248\u672c\u4e2d\uff0c mem.Usage &#8211; mem.Stats[&#8220;cache&#8221;] \u770b\u8d77\u6765\u4e0e\u6211\u4eec\u4e0a\u9762\u5b9e\u73b0\u7684\u65b9\u5f0f\u7c7b\u4f3c\uff0c\u51cf\u6389\u4e86Cache\u90e8\u5206\uff0c\u5177\u4f53\u6e90\u7801\u5982\u4e0b\uff1a \/\/ calculateMemUsageUnixNoCache calculate memory usage of the container. \/\/ Page cache is intentionally excluded to avoid misinterpretation of the output. func calculateMemUsageUnixNoCache(mem [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/posts\/1931"}],"collection":[{"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/comments?post=1931"}],"version-history":[{"count":2,"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/posts\/1931\/revisions"}],"predecessor-version":[{"id":1934,"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/posts\/1931\/revisions\/1934"}],"wp:attachment":[{"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/media?parent=1931"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/categories?post=1931"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/c.ipcpu.com\/wp-json\/wp\/v2\/tags?post=1931"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}