jmxtrans+influxdb+grafana监控zookeeper实战-白红宇

jmxtrans+influxdb+grafana监控zookeeper实战

阅读量：7251 次

发布时间：2019-06-29

本文共 9453 字，大约阅读时间需要 31 分钟。

序

本文主要研究一下如何使用jmxtrans+influxdb+granfa监控zookeeper

配置zookeeper jmx

在conf目录下新增zookeeper-env.sh，并使用chmod +x赋予执行权限，内容如下

JMXLOCALONLY=falseJMXDISABLE=falseJMXPORT=8999JMXAUTH=falseJMXSSL=false

指定JMXPORT为8999

搭建influxdb及grafana

这里使用docker构建，如下

docker network create monitoring-networkdocker run -d -p 8083:8083 -p 8086:8086 \    --net monitoring-network  \    --name influxdb  \    influxdb:1.5.4curl -POST http://192.168.99.100:8086/query --data-urlencode "q=CREATE DATABASE zookeeper"docker run -d -p 3000:3000 --net monitoring-network --name grafana grafana/grafana:5.2.4curl 'http://admin:admin@192.168.99.100:3000/api/datasources' -X POST \-H 'Content-Type: application/json;charset=UTF-8' \--data-binary '{"name":"influx","type":"influxdb","url":"http://192.168.99.100:8086","access":"proxy","isDefault":true,"database":"zookeeper","user":"admin","password":"admin"}'

创建monitoring-network，让docker中的influxdb与grafana连通，然后使用rest api创建zookeeper数据库

创建grafana并指定net为monitoring-network，然后使用rest api配置数据源

jmxtrans配置

下载jmxtrans

wget http://central.maven.org/maven2/org/jmxtrans/jmxtrans/270/jmxtrans-270-all.jar

下载执行脚本

wget -q https://raw.githubusercontent.com/jmxtrans/jmxtrans/master/jmxtrans/jmxtrans.shchmod +x jmxtrans.sh

配置zookeeper.json

{  "servers": [    {      "port": 8999,      "host": "localhost",      "alias": "zk",      "queries": [        {          "outputWriters" : [            {              "@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",              "url" : "http://192.168.99.100:8086/",              "database" : "zookeeper",              "username" : "admin",              "password" : "admin"            } ],          "obj": "org.apache.ZooKeeperService:name0=ReplicatedServer_id*",          "attr": [            "QuorumSize"          ]        },        {          "outputWriters" : [            {              "@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",              "url" : "http://192.168.99.100:8086/",              "database" : "zookeeper",              "username" : "admin",              "password" : "admin"            } ],          "obj": "org.apache.ZooKeeperService:name0=ReplicatedServer_id*,name1=replica*,name2=*",          "attr": [            "MaxClientCnxnsPerHost",            "MaxRequestLatency",            "AvgRequestLatency",            "MinRequestLatency",            "MaxSessionTimeout",            "MinSessionTimeout",            "OutstandingRequests",            "PacketsReceived",            "PacketsSent",            "PendingRevalidationCount",            "TickTime"          ]        },        {          "outputWriters" : [            {              "@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",              "url" : "http://192.168.99.100:8086/",              "database" : "zookeeper",              "username" : "admin",              "password" : "admin"            } ],          "obj": "org.apache.ZooKeeperService:name0=ReplicatedServer_id*,name1=replica*,name2=*,name3=InMemoryDataTree",          "attr": [            "NodeCount",            "WatchCount"          ]        },        {          "outputWriters" : [            {              "@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",              "url" : "http://192.168.99.100:8086/",              "database" : "zookeeper",              "username" : "admin",              "password" : "admin"            } ],          "obj": "org.apache.ZooKeeperService:name0=ReplicatedServer_id*,name1=replica*,name2=*,name3=Connections,name4=*,name5=*",          "resultAlias": "Clients",          "attr": [            "AvgLatency",            "LastLatency",            "MaxLatency",            "MinLatency",            "OutstandingRequests",            "PacketsReceived",            "PacketsSent"          ]        },        {          "outputWriters" : [            {              "@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",              "url" : "http://192.168.99.100:8086/",              "database" : "zookeeper",              "username" : "admin",              "password" : "admin"            } ],          "obj": "org.apache.ZooKeeperService:name0=StandaloneServer_port*",          "resultAlias": "Clients",          "attr": [            "AvgLatency",            "LastLatency",            "MaxLatency",            "MinLatency",            "OutstandingRequests",            "PacketsReceived",            "PacketsSent",            "NumAliveConnections"          ]        },        {          "outputWriters" : [            {              "@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",              "url" : "http://192.168.99.100:8086/",              "database" : "zookeeper",              "username" : "admin",              "password" : "admin"            } ],          "obj": "java.lang:type=Memory",          "resultAlias": "Memory",          "attr": [            "HeapMemoryUsage",            "NonHeapMemoryUsage"          ]        },        {          "outputWriters" : [            {              "@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",              "url" : "http://192.168.99.100:8086/",              "database" : "zookeeper",              "username" : "admin",              "password" : "admin"            } ],          "obj": "java.lang:type=OperatingSystem",          "resultAlias": "Process",          "attr": [            "OpenFileDescriptorCount",            "ProcessCpuLoad"          ]        },        {          "outputWriters" : [            {              "@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",              "url" : "http://192.168.99.100:8086/",              "database" : "zookeeper",              "username" : "admin",              "password" : "admin"            } ],          "obj": "java.lang:type=Threading",          "resultAlias": "Threading",          "attr": [            "ThreadCount"          ]        }      ],      "numQueryThreads": 2    }  ]}

放在跟jmxtrans.sh脚本同一个目录下即可自动加载

启动

JAR_FILE=jmxtrans-270-all.jar ./jmxtrans.sh start

日志输出实例

2018-10-04 11:28:30 [main] level org.quartz.impl.StdSchedulerFactory [StdSchedulerFactory.java:1179] - Using default implementation for ThreadExecutor2018-10-04 11:28:30 [main] level org.quartz.core.SchedulerSignalerImpl [SchedulerSignalerImpl.java:60] - Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl2018-10-04 11:28:30 [main] level org.quartz.core.QuartzScheduler [QuartzScheduler.java:229] - Quartz Scheduler v.1.8.6 created.2018-10-04 11:28:30 [main] level org.quartz.simpl.RAMJobStore [RAMJobStore.java:139] - RAMJobStore initialized.2018-10-04 11:28:30 [main] level org.quartz.core.QuartzScheduler [QuartzScheduler.java:255] - Scheduler meta-data: Quartz Scheduler (v1.8.6) 'ServerScheduler' with instanceId 'Mars.local1538623710548'  Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.  NOT STARTED.  Currently in standby mode.  Number of jobs executed: 0  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.  Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.2018-10-04 11:28:30 [main] level org.quartz.impl.StdSchedulerFactory [StdSchedulerFactory.java:1324] - Quartz scheduler 'ServerScheduler' initialized from an externally opened InputStream.2018-10-04 11:28:30 [main] level org.quartz.impl.StdSchedulerFactory [StdSchedulerFactory.java:1328] - Quartz scheduler version: 1.8.62018-10-04 11:28:30 [main] level org.quartz.core.QuartzScheduler [QuartzScheduler.java:2267] - JobFactory set to: com.googlecode.jmxtrans.guice.GuiceJobFactory@263f04ca2018-10-04 11:28:30 [main] level com.googlecode.jmxtrans.JmxTransformer [JmxTransformer.java:177] - Starting Jmxtrans on : .2018-10-04 11:28:30 [main] level org.quartz.core.QuartzScheduler [QuartzScheduler.java:519] - Scheduler ServerScheduler_$_Mars.local1538623710548 started.2018-10-04 11:28:30 [main] level com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory [InfluxDbWriterFactory.java:121] - Result Tags to write set to: [ResultAttribute(name=typeName), ResultAttribute(name=objDomain), ResultAttribute(name=className), ResultAttribute(name=attributeName)]2018-10-04 11:28:30 [main] level com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory [InfluxDbWriterFactory.java:102] - Connecting to url: http://192.168.99.100:8086/ as: username: admin2018-10-04 11:28:31 [main] level com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory [InfluxDbWriterFactory.java:121] - Result Tags to write set to: [ResultAttribute(name=typeName), ResultAttribute(name=objDomain), ResultAttribute(name=className), ResultAttribute(name=attributeName)]

查询influxdb数据

docker exec -it influxdb influxConnected to http://localhost:8086 version 1.5.4InfluxDB shell version: 1.5.4> use zookeeperUsing database zookeeper> show MEASUREMENTSname: measurementsname----ClientsMemoryProcessThreading> show series from "Memory"key---Memory,attributeName=HeapMemoryUsage,className=sun.management.MemoryImpl,hostname=zk,objDomain=java.lang,typeName=type\=MemoryMemory,attributeName=NonHeapMemoryUsage,className=sun.management.MemoryImpl,hostname=zk,objDomain=java.lang,typeName=type\=Memory