InfluxDB

2018年3月13日 (二) 07:28的最后版本

InfluxDB 是一个 Go 语言开发的开源(MIT)分布式时间序列数据库。

[编辑] TICK堆栈

• Telegraf 数据采集
• InfluxDB 数据存储
• Chronograf 数据可视化
• Kapacitor 监控告警&ETL

注：InfluxDB从0.12版本开始不再将其cluster源码开源。

[编辑] 应用场景

• 定制DevOps监控(Custom DevOps Monitoring)
• 实时分析(Real Time Analytics)
• 物联网和传感器数据(IoT and Sensor Data)
• 云计算(Cloud & OpenStack)
• 异常检测(Anomaly Detection)
• 消息(Messaging)
• 个性化(Personalization)
• 股票交易(Equities Trading)
• 市政基础设施管理(Municipal Infrastructure Management)
• GPS服务(GPS Services)
• 量子物理研究(Quantum Physics Research)
• POS系统(Point of Sale Systems)
• 制造业和家庭自动化(Manufacturing & Home Automation)
• 运输和物流(Transportation & Material Logistics)

[编辑] 指南

OS X 安装

$ brew update
$ brew install influxdb
$ brew install telegraf // 数据采集
$ brew install homebrew/binary/chronograf // 数据可视化
$ brew install kapacitor // 告警、数据处理
$ influxd
$ influx
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 0.9.6.1
InfluxDB shell 0.9.6.1
> create database mydb
> show databases
> use mydb
> insert cpu,host=serverA,region=us_west value=0.64
> select * from cpu
> insert temperature,machine=unit42,type=assembly external=25,internal=37
> select * from temperature
> show series
> show measurements
> show tag keys
> show field keys

创建数据库

curl -G 'http://localhost:8086/query' --data-urlencode "q=CREATE DATABASE mydb"

插入数据

curl -XPOST 'http://localhost:8086/write?db=mydb' -d 'cpu,host=server01,region=uswest load=42 1434055562000000000'
curl -XPOST 'http://localhost:8086/write?db=mydb' -d 'cpu,host=server02,region=uswest load=78 1434055562000000000'
curl -XPOST 'http://localhost:8086/write?db=mydb'-d 'cpu,host=server03,region=useast load=15.4 1434055562000000000'

查询数据

curl -G http://localhost:8086/query?pretty=true --data-urlencode "db=mydb" --data-urlencode "q=SELECT * FROM cpu WHERE host='server01' AND time < now() - 1d"

分析数据

curl -G http://localhost:8086/query?pretty=true --data-urlencode "db=mydb" --data-urlencode "q=SELECT mean(load) FROM cpu WHERE region='uswest'"

sample数据集

curl https://s3-us-west-1.amazonaws.com/noaa.water.database.0.9/NOAA_data.txt > sample-data.txt
influx -import -path=sample-data.txt -precision=s

提示信息: 2016/02/23 10:32:35 Processed 76290 inserts

> use NOAA_water_database
> show measurements
> select count(water_level) from h2o_feet
> select * from h2o_feet limit 5

Web管理接口

http://localhost:8083/

创建管理员

> CREATE USER admin WITH PASSWORD 'admin'
> GRANT ALL PRIVILEGES TO admin
> show users
user  admin
----  -----
hupo  false
admin true

[编辑] Grafana

Grafana 是面向 Graphite, InfluxDB & OpenTSDB 可视化仪表盘。

[编辑] Telegraf

Telegraf数据采集，一个插件驱动的服务器代理。

$ telegraf --help
$ telegraf -sample-config -input-filter cpu -output-filter influxdb

[编辑] Chronograf

$ Chronograf
http://127.0.0.1:10000

[编辑] Kapacitor

启动Kapacitor

kapacitord config > kapacitor.conf
kapacitord -config kapacitor.conf
http://localhost:9092/

Kapacitor 使用 TICKscript DSL 来定义任务(tasks)。

如：cpu_alert.tick

stream
   // 从sample数据库获得cpu测量值
   .from().measurement('cpu')
   .alert()
       .crit(lambda: "usage_idle" <  70)
       // 写入日志文件
       .log('/tmp/alerts.log')

写入task

kapacitor define \
   -name cpu_alert \
   -type stream \
   -tick cpu_alert.tick \
   -dbrp mydb.default

运行task

kapacitor record stream -name cpu_alert -duration 20s

获取数据

kapacitor list recordings $rid 

返回类似数据

ID                                      Type    Size      Created                
32bc3657-4a96-472e-a982-11afc1bfb51a    stream  0 B       23 Feb 16 17:42 CST    
ff90d6c3-f4a5-441c-b69d-7b23ee753398    stream  0 B       23 Feb 16 17:48 CST   

Kapacitor控制台返回类似数据

[httpd] 2016/02/23 17:49:57 I! ::1 - - [23/Feb/2016:17:49:57 +0800] GET /recordings?rids= HTTP/1.1 200 173 - Go-http-client/1.1 cc17e978-da12-11e5-8019-000000000000 362.8µs
[httpd] 2016/02/23 17:50:10 I! ::1 - - [23/Feb/2016:17:50:10 +0800] GET /recordings?rids= HTTP/1.1 200 173 - Go-http-client/1.1 d42f803b-da12-11e5-801a-000000000000 1.570713ms
[httpd] 2016/02/23 17:50:30 I! ::1 - - [23/Feb/2016:17:50:30 +0800] GET /recordings?rids= HTTP/1.1 200 173 - Go-http-client/1.1 dfc64c93-da12-11e5-801b-000000000000 1.266153ms

[编辑] Relay

• Relay @ GitHub
• Introducing InfluxDB Relay: Open Source High-Availability for InfluxDB

[编辑] 存储

• The New InfluxDB Storage Engine: Time Structured Merge Tree

[编辑] 比较

• InfluxDB Markedly Outperforms OpenTSDB in Time-Series Data & Metrics Benchmark
• InfluxDB Tops Cassandra in Time-Series Data & Metrics Benchmark
• InfluxDB is 27x Faster vs MongoDB for Time-Series Workloads
• InfluxDB Markedly Outperforms Elasticsearch in Time-Series Data & Metrics Benchmark

[编辑] 语言

InfluxDB支持的各种编程语言

• Introducing the ReactiveInflux: non-blocking InfluxDB driver for Scala and Java supporting Apache Spark

[编辑] 用户

• 七牛从InfluxDB看时序数据的处理
• 滴滴打车
• Mirantis OpenStack

更多用户>>>

[编辑] 图集

• 

  架构

• 

  TICK堆栈

• 

  TICK堆栈

• 

  Relay高可用

• 

  IoT

• 

  OpenStack告警日志指标管理

• 

  实时分析

• 

  DevOps监控

• 

  WAL(Write Ahead Log)

• 

  Web管理接口

• 

  Chronograf数据可视化

[编辑] 链接

• InfluxDB官网
• InfluxDB @ GitHub
• InfluxData Blog
• How to send sensor data to InfluxDB from an Arduino Uno
• How-to Create an IoT Project with the TICK Stack on the Google Cloud Platform
• 使用 StatsD + InfluxDB + Grafana 搭建 Node.js 监控系统 (一)
• 使用 StatsD + InfluxDB + Grafana 搭建 Node.js 监控系统 (二)