微信公众号

使用diamond监控系统或api

引言

1
2
如何保证系统或api在出问题的时候及时发现并解决呢?这就需要用到监控系统,监控系统可以是实时监控,也可以是
非实时的(需要定时去检查系统),这里说一下用diamond实现非实时监控

安装

1
2
# cd /usr/local/Diamond
# git clone git@github.com:python-diamond/Diamond.git

配置

diamond配置(./conf/diamond.conf)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
[server]

handlers = diamond.handler.graphite.GraphiteHandler, diamond.handler.archive.ArchiveHandler

# User diamond will run as
# Leave empty to use the current user
user =

# Group diamond will run as
# Leave empty to use the current group
group =

# Pid file
pid_file = /xx/Diamond/run/diamond.pid

# Directory to load collector modules from
collectors_path = /xx/Diamond/share/diamond/collectors/
collectors_config_path = /xx/Diamond/conf/collectors/
collectors_reload_interval = 5

# Directory to load handler configs from
handlers_config_path = /xx/Diamond/conf/handlers/

# Directory to load handler modules from
handlers_path = /xx/Diamond/share/diamond/handlers/

################################################################################
### Options for handlers
[handlers]

# daemon logging handler(s)
keys = rotated_file

### Defaults options for all Handlers
[[default]]

[[ArchiveHandler]]

# File to write archive log files
log_file = /var/log/diamond/archive.log

# Number of days to keep archive log files
days = 7

[[GraphiteHandler]]
### Options for GraphiteHandler

# Graphite server host(本人认为携程carbon比较合适)
host = xx.xx.xx.xx

# Port to send metrics to
port = 2003

# Socket timeout (seconds)
timeout = 15

# Batch size for metrics
batch = 1

[[GraphitePickleHandler]]
### Options for GraphitePickleHandler

# Graphite server host(本人认为携程carbon比较合适)
host = xx.xx.xx.xx #写你搭建好的carbon服务器地址

# Port to send metrics to
port = 2004

# Socket timeout (seconds)
timeout = 15

# Batch size for pickled metrics
batch = 256

[[MySQLHandler]]
### Options for MySQLHandler

# MySQL Connection Info
hostname = xx.xx.xx.xx
port = xx
username = xx
password = xx
database = diamond
table = metrics
# INT UNSIGNED NOT NULL
col_time = timestamp
# VARCHAR(255) NOT NULL
col_metric = metric
# VARCHAR(255) NOT NULL
col_value = value

[[StatsdHandler]]
host = 127.0.0.1
port = 8125

[[TSDBHandler]]
host = 127.0.0.1
port = 4242
timeout = 15

[[LibratoHandler]]
user = user@example.com
apikey = abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz01

[[HostedGraphiteHandler]]
apikey = abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz01
timeout = 15
batch = 1

# And any other config settings from GraphiteHandler are valid here

[[HttpPostHandler]]

### Urp to post the metrics
url = http://localhost:8888/
### Metrics batch size
batch = 100


################################################################################
### Options for collectors
[collectors]
[[default]]
### Defaults options for all Collectors

# Uncomment and set to hardcode a hostname for the collector path
# Keep in mind, periods are seperators in graphite
# hostname = my_custom_hostname

# If you prefer to just use a different way of calculating the hostname
# Uncomment and set this to one of these values:

# smart = Default. Tries fqdn_short. If that's localhost, uses hostname_short

# fqdn_short = Default. Similar to hostname -s
# fqdn = hostname output
# fqdn_rev = hostname in reverse (com.example.www)

# uname_short = Similar to uname -n, but only the first part
# uname_rev = uname -r in reverse (com.example.www)

# hostname_short = `hostname -s`
# hostname = `hostname`
# hostname_rev = `hostname` in reverse (com.example.www)

# shell = Run the string set in hostname as a shell command and use its
# output(with spaces trimmed off from both ends) as the hostname.

# hostname_method = smart

# Path Prefix and Suffix
# you can use one or both to craft the path where you want to put metrics
# such as: %(path_prefix)s.$(hostname)s.$(path_suffix)s.$(metric)s
path_prefix = servers #在graphite显示的最外层名字
# path_suffix =

# Path Prefix for Virtual Machines
# If the host supports virtual machines, collectors may report per
# VM metrics. Following OpenStack nomenclature, the prefix for
# reporting per VM metrics is "instances", and metric foo for VM
# bar will be reported as: instances.bar.foo...
# instance_prefix = instances

# Default Poll Interval (seconds)
interval = 10 #(所有的监控多长时间执行一次,并发送数据给carbon)

###############################################
# Default enabled collectors
########################################

#[[CPUCollector]]
#enabled = True

#[[DiskSpaceCollector]]
#enabled = True

#[[DiskUsageCollector]]
#enabled = True

#[[LoadAverageCollector]]
#enabled = True

[[MemoryCollector]]
enabled = True

#[[VMStatCollector]]
#enabled = True

[loggers]

keys = root

# handlers are higher in this config file, in:
# [handlers]
# keys = ...

[formatters]

keys = default

[logger_root]

# to increase verbosity, set DEBUG
level = INFO
handlers = rotated_file
propagate = 1

[handler_rotated_file]

class = handlers.TimedRotatingFileHandler
level = DEBUG
formatter = default
# rotate at midnight, each day and keep 7 days
args = ('/var/log/diamond/diamond.log', 'midnight', 1, 7)

[formatter_default]

format = [%(asctime)s] [%(threadName)s] %(message)s
datefmt =

#######################################
### Options for config merging
# [configs]
# path = "/etc/diamond/configs/"
# extension = ".conf"
#----------------------------------------------
# Example:
# /etc/diamond/configs/net.conf
# [collectors]
#
# [[NetworkCollector]]
# enabled = True

数据库支持(mysql)

1
create database diamond

增加MemoryCollector(当然按照上边的配置还能加其他的)

1
2
3
在目录/xx/Diamond/share/diamond/collectors/中添加一个py文件
该文件diamond已经写好,在src目录下寻找一下
(src/collectors/memory/memory.py)

增加自己的Collector

添加ExampleCollector.conf

1
2
在/xx/Diamond/conf/collectors/添加一个ExampleCollector.conf
(你也可以是其他的名字,xxCollector.conf )

添加ExampleCollector.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
在/xx/Diamond/share/diamond/collectors/中添加ExampleCollector.py
# coding=utf-8
import diamond.collector


def test():
# dosomething
pass

class ExampleCollector(diamond.collector.Collector):

def collect(self):
metric_name = "my.example.metric"
st = time.time()
test()
et = time.time()
metric_value = (et - st) * 1000
self.publish(metric_name, metric_value)

if __name__ == '__main__':
pass

启动

1
2
# cd /usr/local/Diamond
# python ./bin/diamond -lf -c ./conf/diamond.conf

查看结果

1
2
可以到已经搭建好的graphite(待我整理完会补充,也可以到网上搜一下)
后台查看应该会有 /Metrics/servers .....

More info: Diamond

yunsonbai wechat
微信公众号