Apache Cassandra

来自开放百科 - 灰狐
（版本间的差异）
跳转到： 导航, 搜索
 
小 (→‎项目)
 
（未显示2个用户的148个中间版本）
第1行： 第1行：
−
Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store. Cassandra brings together the distributed systems technologies from Dynamo and the data model from Google's BigTable. Like Dynamo, Cassandra is [http://www.allthingsdistributed.com/2008/12/eventually_consistent.html eventually consistent]. Like BigTable, Cassandra provides a ColumnFamily-based data model richer than typical key/value systems.
+
{{SeeWikipedia}}
  
−
==Links==
+
Apache Cassandra是一套开源分布式Key-Value存储系统。它最初由[[Facebook]]开发，用于储存特别大的数据。Facebook目前在使用此系统。
−
*http://incubator.apache.org/cassandra/
+
[[文件:cassandra.png|right]]
 +
[[文件:datastax.png|right]]
 +
 
 +
==理论基础==
 +
*peer-to-peer、环形架构基于 Amazon [[Dynamo]]
 +
*数据存储模型基于 Google [[BigTable]]
 +
*[http://www.planetcassandra.org/blog/cassandra-daughter-of-dynamo-and-bigtable/ Cassandra: Daughter of Dynamo and BigTable]
 +
*[[gossip protocol]]
 +
*[[Staged event-driven architecture|SEDA]]
 +
 
 +
==主要特性==
 +
* 分布式
 +
* 基于Column的结构化
 +
* 高度可伸展性
 +
2 nodes can handle 100,000 transactions per second, 4 nodes will support 200,000 transactions/sec and 8 nodes will tackle 400,000 transactions/sec。
 +
 
 +
[[文件:cassandra-supplies-linear-scalability.png]]
 +
 
 +
Cassandra的主要特点就是它不是一个数据库，而是由一堆数据库节点共同构成的一个分布式网络服务，对Cassandra 的一个写操作，会被复制到其他节点上去，对Cassandra的读操作，也会被路由到某个节点上面去读取。对于一个Cassandra群集来说，扩展性能是比较简单的事情，只管在群集里面添加节点就可以了。
 +
 
 +
Cassandra是一个混合型的非关系型数据库，类似于Google的[[BigTable]]。其主要功能比 [[Dynomite]]（分布式的Key-Value存储系统）更丰富，但支持度却不如文档存储[[MongoDB]]（介于关系数据库和非关系数据库之间的开源产品，是非关系数据库当中功能最丰富，最像关系型数据库。支持的数据结构非常松散，是类似[[JSON]]的bjson格式，因此可以存储比较复杂的数据类型）Cassandra最初由Facebook开发，后转变成了开源项目。它是一个网络社交云计算方面理想的数据库。以Amazon专有的完全分布式的[[Dynamo]]为基础，结合了Google BigTable基于列族（Column Family）的数据模型。P2P去中心化的存储。很多方面都可以称之为Dynamo 2.0。
 +
 
 +
和其他数据库比较，有几个突出特点：
 +
*模式灵活：使用Cassandra，像文档存储，你不必提前解决记录中的字段。你可以在系统运行时随意的添加或移除字段。这是一个惊人的效率提升，特别是在大型部署上。
 +
*真正的可扩展性：Cassandra是纯粹意义上的水平扩展。为给集群添加更多容量，可以指向另一台电脑。你不必重启任何进程，改变应用查询，或手动迁移任何数据。
 +
*多数据中心识别：你可以调整你的节点布局来避免某一个数据中心起火，一个备用的数据中心将至少有每条记录的完全复制。
 +
 
 +
一些使Cassandra提高竞争力的其他功能：
 +
*范围查询：如果你不喜欢全部的键值查询，则可以设置键的范围来查询。
 +
*列表数据结构：在混合模式可以将超级列添加到5维。对于每个用户的索引，这是非常方便的。
 +
*分布式写操作：有可以在任何地方任何时间集中读或写任何数据。并且不会有任何单点失败。
 +
 
 +
==版本==
 +
*[http://docs.huihoo.com/apache/cassandra/4.x/doc/latest/ 4.x]
 +
*3.x：[http://docs.huihoo.com/apache/cassandra/3.9/html/ 3.9], [http://docs.huihoo.com/javadoc/apache/cassandra/3.9/ 3.9 javadoc]
 +
*2.x
 +
[http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-2-json-support What’s New in Cassandra 2.2: JSON Support]
 +
*1.x
 +
 
 +
==指南==
 +
===OS X===
 +
brew install cassandra
 +
brew info cassandra
 +
cassandra -f
 +
 
 +
bin/cassandra -f
 +
bin/cqlsh
 +
or cqlsh 1.2.3.4 9042 // ip, port
 +
cqlsh> help
 +
cqlsh> describe keyspaces;
 +
cqlsh> use system;
 +
cqlsh:system> select * from schema_keyspaces; // 所有keyspaces
 +
cqlsh:system> describe schema_keyspaces; // schema_keyspaces所有表和表定义
 +
cqlsh> CREATE KEYSPACE mykeyspace
 +
    ... WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
 +
cqlsh> use mykeyspace;
 +
cqlsh:mykeyspace> CREATE TABLE users (
 +
              ... user_id int PRIMARY KEY,
 +
              ... fname text,
 +
              ... lname text
 +
              ... );
 +
cqlsh:mykeyspace> INSERT INTO users (user_id,  fname, lname)
 +
              ... VALUES (1745, 'john', 'smith');
 +
cqlsh:mykeyspace> INSERT INTO users (user_id,  fname, lname)
 +
              ... VALUES (1744, 'john', 'doe');
 +
cqlsh:mykeyspace> INSERT INTO users (user_id,  fname, lname)
 +
              ... VALUES (1746, 'john', 'smith');
 +
cqlsh:mykeyspace> select * from users;
 +
 
 +
  user_id | fname | lname
 +
---------+-------+-------
 +
    1745 |  john | smith
 +
    1744 |  john |  doe
 +
    1746 |  john | smith
 +
 
 +
cqlsh:mykeyspace> select now(), uuid(), token() from mykeyspace.users;
 +
 
 +
You have to be logged in and not anonymous to perform this request
 +
vim cassandra.yaml
 +
authenticator: PasswordAuthenticator
 +
创建新用户
 +
cassandra@cqlsh> create role huihoo with superuser = true and login = true and password = 'huihoo';
 +
cassandra@cqlsh> list roles;
 +
 
 +
===Tools===
 +
[http://docs.datastax.com/en/cassandra/2.2/cassandra/tools/toolsCStress.html cassandra-stress tool]
 +
cd apache-cassandra-2.1.8/tools/bin
 +
cassandra-stress help -schema
 +
cassandra-stress write n=1000000 // 写100万行
 +
cassandra-stress read n=200000 // 读20万行
 +
cassandra-stress write n=1000000 cl=one -mode native cql3 -schema keyspace="stress" -log file=~/load_1M_rows.log // 写入100万行
 +
cqlsh:mykeyspace> select count(*) from stress.standard1;
 +
  count
 +
---------
 +
  1000000
 +
cqlsh> describe stress.standard1;
 +
cqlsh> select * from stress.standard1 limit 10;
 +
 
 +
===系统信息===
 +
nodetool --host 127.0.0.1 cfstats
 +
cqlsh> describe system;
 +
cqlsh> select * from system.batchlog;
 +
cqlsh> select * from system.compaction_history;
 +
cqlsh> select * from system.compactions_in_progress;
 +
cqlsh> select * from system.hints;
 +
cqlsh> select * from system.local;
 +
cqlsh> select * from system.peers;
 +
cqlsh> select * from system.peer_events;
 +
cqlsh> select * from system.range_xfers;
 +
cqlsh> select * from system.sstable_activity;
 +
cqlsh> select * from system.schema_columnfamilies;
 +
cqlsh> select * from system.schema_columns;
 +
cqlsh> select * from system.schema_triggers;
 +
cqlsh> select * from system.schema_usertypes;
 +
cqlsh> select * from system.size_estimates;
 +
cqlsh> select * from system.schema_keyspaces;
 +
cqlsh> select * from mykeyspace.users;
 +
cqlsh> select * from system_traces.sessions;
 +
cqlsh> select * from system_traces.events;
 +
 
 +
==CQL==
 +
*[https://cassandra.apache.org/doc/cql3/ Cassandra Query Language (CQL) v3]
 +
*[http://docs.huihoo.com/apache/cassandra/planetcassandra/CQL-This-is-not-he-SQL-you-are-looking-for.pdf CQL: This is not the SQL you are looking for]
 +
*[http://docs.huihoo.com/apache/cassandra/datastax/CQL-3.1-for-Cassandra-2.0-and-2.1.pdf CQL for Cassandra 2.x]
 +
 
 +
[http://www.planetcassandra.org/blog/user-defined-functions-in-cassandra-3-0/ CQL and Java type comparison]
 +
CQL              Java
 +
===              ====
 +
boolean          java.lang.Boolean
 +
int              java.lang.Integer
 +
bigint          java.lang.Long
 +
float            java.lang.Float
 +
double          java.lang.Double
 +
inet            java.net.InetAddress
 +
text            java.lang.String
 +
ascii            java.lang.String
 +
timestamp        java.util.Date
 +
uuid            java.util.UUID
 +
timeuuid        java.util.UUID
 +
varint          java.math.BigInteger
 +
decimal          java.math.BigDecimal
 +
blob            java.nio.ByteBuffer
 +
list<E>          java.util.List<E>      where E is also a type from this list
 +
set<E>          java.util.Set<E>      where E is also a type from this list
 +
map<K,V>        java.util.Map<K,V>    where K and V is also a types from this list
 +
(user type)      com.datastax.driver.core.UDTValue
 +
(tuple type)    com.datastax.driver.core.TupleValue
 +
 
 +
==项目==
 +
[https://github.com/topics/cassandra Apache Cassandra Topics]
 +
*[https://github.com/strapdata/elassandra Elassandra] = [[Elasticsearch]] + Apache Cassandra
 +
*[https://github.com/OpenNMS/newts Newts] is a time-series data store based on Apache Cassandra.
 +
 
 +
==C++==
 +
[[ScyllaDB]] 是用 [[C++]] 重写的 Apache Cassandra，完全兼容 Cassandra.
 +
 
 +
==.NET==
 +
*[https://github.com/datastax/csharp-driver DataStax C# Driver for Apache Cassandra]
 +
 
 +
==Python==
 +
*[https://github.com/twissandra/twissandra Twissandra]
 +
*[https://github.com/rustyrazorblade/python-presentation Python & Cassandra - Best Friends]
 +
sudo pip install ipython-cql
 +
sudo pip install virtualenvwrapper
 +
source /usr/local/bin/virtualenvwrapper.sh
 +
git clone https://github.com/rustyrazorblade/python-presentation
 +
cd python-presentation
 +
mkvirtualenv tutorial
 +
pip install -r requirements.txt
 +
ipython notebook
 +
 
 +
==MariaDB==
 +
*[http://www.infoq.com/cn/news/2012/11/Cassandra-SE MariaDB的Cassandra存储引擎] 允许[[MariaDB]]通过标准SQL语法使用Cassandra集群。
 +
*[https://mariadb.com/kb/en/mariadb/cassandra/ Cassandra Storage Engine]
 +
 
 +
==Docker==
 +
*[http://docs.huihoo.com/datastax/Best-Practices-for-Running-DataStax-Enterprise-within-Docker.pdf Best Practices for Running DataStax Enterprise within Docker]
 +
 
 +
==[[Apache Spark|Spark]]==
 +
*[http://www.datastax.com/dev/blog/zen-art-spark-maintenance Zen and the Art of Spark Maintenance]
 +
*[https://github.com/datastax/spark-cassandra-connector DataStax Spark Cassandra Connector]
 +
*[https://tobert.github.io/post/2014-07-15-installing-cassandra-spark-stack.html Installing the Cassandra / Spark OSS Stack]
 +
 
 +
==[[Apache Hadoop|Hadoop]]==
 +
[https://wiki.apache.org/cassandra/HadoopSupport Hadoop Support]
 +
 
 +
==[[Presto]]==
 +
[https://prestodb.io/docs/current/connector/cassandra.html Cassandra Connector]
 +
 
 +
==集群HA==
 +
*[https://github.com/riptano/ccm CCM (Cassandra Cluster Manager)]
 +
*[https://github.com/mesosphere/cassandra-mesos Cassandra on Mesos]
 +
 
 +
==控制台==
 +
*[https://github.com/Netflix/Priam Priam (Netflix)]
 +
*[https://github.com/suguru/cassandra-webconsole cassandra-webconsole]
 +
*[https://github.com/sebgiroux/Cassandra-Cluster-Admin Cassandra Cluster Admin]
 +
*[https://github.com/hmsonline/virgil Virgil is a services layer for Cassandra]
 +
*[https://github.com/tbarbugli/cassandra_snapshotter Cassandra Snapshotter]
 +
*[https://github.com/hailocab/ctop CTOP ("Top for Cassandra")]
 +
 
 +
==IDE==
 +
[https://cassandra.apache.org/doc/latest/development/ide.html Building and IDE Integration]
 +
*[https://downloads.datastax.com/#devcenter DataStax DevCenter] 已不再更新维护 [https://github.com/Patipat-Chuensuwannakul/devcenter @ GitHub]
 +
*[https://downloads.datastax.com/#desktop DataStax Desktop]
 +
*[https://downloads.datastax.com/#studio DataStax Studio] Studio only supports DataStax Enterprise clusters. 
 +
$ ./bin/server.sh
 +
http://127.0.0.1:9091
 +
$ pkill -f studio
 +
*[https://hackolade.com Hackolade]
 +
 
 +
==Operator==
 +
*[https://github.com/instaclustr/cassandra-operator Kubernetes Operator for Cassandra]
 +
 
 +
==驱动==
 +
*[http://www.planetcassandra.org/apache-cassandra-client-drivers/ Apache Cassandra Client Drivers]
 +
*[https://github.com/Netflix/astyanax Astyanax] is a high level Java client for Apache Cassandra
 +
*[https://pypi.python.org/pypi/django-cassandra-engine/ django-cassandra-engine] - the Cassandra backend for [[Django]]
 +
*[https://github.com/impetus-opensource/Kundera Kundera]
 +
*pip install cassandra-driver // python
 +
 
 +
==厂商==
 +
*[[DataStax]] 下载 [http://www.planetcassandra.org/cassandra/ DataStax Community Edition — Apache Cassandra]
 +
*[http://www.satellitevolta.com/ Satellite Volta]
 +
*[http://www.seastar.io/ Seastar]
 +
 
 +
==案例==
 +
*[http://cio.zdnet.com.cn/cio/2013/0513/2159184.shtml Netflix用Apache Cassandra替代甲骨文数据库]
 +
目前，Cassandra对于[https://netflix.com Netflix]而言是首选数据库，因为它们几乎满足了Netflix的所有需求。Netflix已经将95%的数据存储在Cassandra上，包括客户账户信息、影片评分、影片元数据、影片书签和日志等。Netflix在750多个节点上运行着50多个Cassandra集群。高峰时，Netflix每秒要处理50,000多个读取和100,000写入操作。Netflix平均每天要处理21亿次的读取与43亿次的写入操作。
 +
*[[DataStax]]也为其他各种行业建立了不同版本的Cassandra工具。DataStax已经筹资8400万美元，目前有员工300多人，正准备IPO。埃利斯称，他们已经有500多家客户，包括“财富100强”中的25家大公司。
 +
*[http://tech.qq.com/a/20140807/006287.htm 这款数据库曾被Facebook抛弃 现正帮苹果壮大] Apple's, with over 75,000 nodes storing over 10 PB of data.
 +
*[http://www.planetcassandra.org/blog/post/soundcloud-activity-feed-and-real-time-stats-powered-by-apache-cassandra SoundCloud's Activity Feed and Real-Time Stats Powered by Apache Cassandra]
 +
*[http://www.planetcassandra.org/blog/interview/facebooks-instagram-making-the-switch-to-cassandra-from-redis-a-75-insta-savings/ Facebook’s Instagram: Making the Switch to Cassandra from Redis, a 75% ‘Insta’ Savings]
 +
*[http://www.planetcassandra.org/blog/post/nexgate-choses-cassandra-over-mongodb-for-a-multi-master-nosql-solution Nexgate Chooses Cassandra over MongoDB for their Multi-Master NoSQL Solution]
 +
*[http://www.planetcassandra.org/blog/post/urban-airship-utilizing-cassandra-to-connect-hundreds-of-millions-of-devices-for-breaking-news-alerts Urban Airship Utilizing Cassandra to Connect Hundreds of Millions of Devices for Breaking News Alerts]
 +
*[http://www.planetcassandra.org/blog/post/apache-cassandra-powers-yakaz-for-10-million-unique-visitors-every-month Apache Cassandra Powers Yakaz for 10 Million Unique Visitors Every Month]
 +
*[http://www.planetcassandra.org/blog/post/i2o-water-switches-to-cassandra-from-microsoft-sql-server-saving-over-100-million-liters-of-water-per-day-across-the-world i2O Water Switches to Cassandra from Microsoft SQL Server, Saving Over 100 Million Liters of Water per Day Across the World]
 +
*[http://www.planetcassandra.org/blog/post/appdynamics-utilizes-cassandra-for-app-monitoring-and-metrics-tracking AppDynamics Utilizes Cassandra for App Monitoring and Metrics Tracking]
 +
*[https://tech.coursera.org/blog/2014/09/23/courseras-adoption-of-cassandra/ Coursera’s Adoption of Cassandra]
 +
*[http://www.infoq.com/cn/articles/spotify-migrate-cassandra Spotify是怎样从Postgres切换至Cassandra的？] [http://www.planetcassandra.org/blog/personalization-at-spotify-using-apache-cassandra/ Personalization at Spotify Using Apache Cassandra], [http://www.planetcassandra.org/blog/interview/spotify-scales-to-the-top-of-the-charts-with-apache-cassandra-at-40k-requestssecond/ Spotify scales to the top of the charts with Apache Cassandra at 40k requests/second]
 +
*[http://www.planetcassandra.org/blog/interview/fighting-fraud-at-nosql-scale-rsa-migrates-from-oracle-to-apache-cassandra-to-protect-your-online-banking/ RSA migrates from Oracle to Apache Cassandra to protect your online banking]
 +
*[http://www.planetcassandra.org/blog/interview/gaming-dev-platform-unity-powers-up-with-cassandra-migrates-away-from-mongodb-for-a-scalable-low-latency-solution/ Gaming dev platform Unity powers up with Cassandra; migrates away from MongoDB for a scalable low latency solution]
 +
*[http://www.planetcassandra.org/blog/interview/coursera-migrates-to-the-top-of-the-class-moves-to-cassandra-for-an-always-on-on-demand-classroom/ Coursera migrates to the top of the class; moves their over 9 million students to Cassandra for an always on, on-demand classroom]
 +
*[http://www.planetcassandra.org/blog/interview/multimedia-messaging-app-cubie-is-ready-to-grow-worry-free-with-apache-cassandra/ Multimedia messaging app Cubie is ready to grow, worry free, with Apache Cassandra]
 +
*[http://www.planetcassandra.org/blog/multi-datacenter-cassandra-on-32-raspberry-pis/ Multi-Datacenter Cassandra on 32 Raspberry Pi’s]
 +
 
 +
==迁移==
 +
数据库迁移
 +
*[http://www.planetcassandra.org/mysql-to-cassandra-migration/ MySQL to Cassandra Migrations]
 +
*[http://www.planetcassandra.org/oracle-to-cassandra-migration/ Oracle to Cassandra Migrations]
 +
*[http://www.planetcassandra.org/mongodb-to-cassandra-migration/ MongoDB to Cassandra Migrations]
 +
*[http://www.planetcassandra.org/hbase-to-cassandra-migration/ HBase to Cassandra Migrations]
 +
*[http://www.planetcassandra.org/redis-to-cassandra-migration/ Redis to Cassandra Migrations]
 +
 
 +
==文档==
 +
*[http://docs.huihoo.com/apache/spark/summit/east2015/SSE15-11-Delivering-Meaning-At-High-Velocity-with-Spark-Streaming-Cassandra-Kafka-and-Akka.pdf Delivering Meaning In NearReal Time At High Velocity & Massive Scale]
 +
*[http://docs.huihoo.com/oreilly/conferences/strataconf/big-data-conference-ny-2013/An-Introduction-to-Real-Time-Analytics-with-Cassandra-and-Hadoop.pdf An Introduction to Real-Time Analytics with Cassandra and Hadoop]
 +
*[http://docs.huihoo.com/apache/cassandra/Cassandra-Data-Modeling-Best-Practices-at-eBay.pdf Cassandra Data Modeling Best Practices at eBay] [http://docs.huihoo.com/apache/cassandra/Cassandra-at-eBay.pdf Cassandra at eBay] [http://docs.huihoo.com/apache/cassandra/planetcassandra/Cassandra-at-eBay-Scale.pdf Cassandra Scale at eBay]
 +
 
 +
==图集==
 +
<gallery>
 +
image:cassandra-column.png|Column
 +
image:cassandra-row.png|Row
 +
image:cassandra-column-family.png|Column family
 +
image:cassandra-keyspace.png|Keyspace
 +
image:cassandra-super-column-and-super-column-family.png|Super
 +
image:Cassandra-File-System-CFS.png|CFS文件系统
 +
image:cassandra-virtual-nodes.png|虚拟节点
 +
image:Spark-Streaming-Cassandra-Kafka-and-Akka.png|Cassandra+Kafka+Akka
 +
image:opscenter-architecture.png|OpsCenter架构
 +
image:Datastax-DevCenter.png|DevCenter
 +
image:recommendation-engine-personalization.png|个性化推荐引擎
 +
image:RabbitMQ-Storm-Cassandra.png|RabbitMQ-Storm-Cassandra实时分析
 +
image:RabbitMQ-Storm-Esper-Cassandra.png|实时分析集成Esper
 +
image:Microservice-platform.png|微服务平台
 +
image:cassandra-write-path.png|写路径
 +
image:cassandra-read-path.png|读路径
 +
image:traditional-and-cassandra-data-modeling.png|关系型和Cassandra
 +
image:tables-in-cassandra.png|Cassandra中的表
 +
image:relational-model-vs-cassandra-model.png|与关系型对比
 +
image:ebay-user-cassandra-model.png|eBay用户模型
 +
image:ebay-cassandra-data-model.png|eBay用户模型
 +
image:cassandra-distributed-Index.png|分布式索引
 +
image:Cassandra-multiple-DC-Consistency-LocalOne.png|多数据中心一致性
 +
image:cassandra-anti-entropy-repair.png|Gossip反熵算法
 +
image:cassandra-keyspace-02.png|Keyspace
 +
image:cassandra-keyspace-02-multi-dc.png|Keyspace多数据中心
 +
image:cassandra-and-spark-analysis-data-center.png|Cassandra+Spark分析数据中心
 +
image:jconsole-cassandra.png|JConsole
 +
image:Gartner-Magic-Quadrant-for-Operational-Database-Management-Systems-October-2015.png|Gartner魔力象限
 +
image:A-simple-hotel-search-system-using-RDBMS.png|RDBMS模型
 +
image:The-hotel-search-represented-with-Cassandra-model.png|Cassandra模型
 +
image:mariadb-cassandra-storage-engine.png|MariaDB存储引擎
 +
image:32-node-raspberry-pi-cassandra-cluster-01.jpg|Cassandra集群
 +
image:32-node-raspberry-pi-cassandra-cluster-04.jpg|Cassandra集群
 +
image:32-node-raspberry-pi-cassandra-cluster-02.jpg|Cassandra集群
 +
image:32-node-raspberry-pi-cassandra-cluster-03.jpg|Cassandra集群
 +
image:Cassandra-Kubernetes-Operator.jpg|K8s运营
 +
</gallery>
 +
 
 +
==链接==
 +
*[http://cassandra.apache.org/ Apache Cassandra官方网站]
 +
*[https://github.com/apache/cassandra Cassandra @ GitHub]
 +
*[https://www.datastax.com/blog DataStax Blog]
 +
*[http://docs.huihoo.com/apache/cassandra/ Apache Cassandra文档]
 +
*[http://docs.datastax.com/en/ DataStax Cassandra文档]
 +
*[https://www.ibm.com/developerworks/cn/opensource/os-apache-cassandra/ 考虑 Apache Cassandra 数据库]
 +
*[http://www.infoq.com/cn/articles/best-practice-of-cassandra-data-model-design Cassandra数据模型设计最佳实践（上部）]
 +
*[http://www.infoq.com/cn/articles/best-practices-cassandra-data-model-design-part2 Cassandra数据模型设计最佳实践（下部）]
 +
*[http://www.planetcassandra.org/blog/the-most-important-thing-to-know-in-cassandra-data-modeling-the-primary-key/ The most important thing to know in Cassandra data modeling: The primary key]
 +
*[http://www.infoq.com/cn/articles/cassandra-2nd-edition-book-review 《Cassandra权威指南》第二版书评及访谈]
 +
 
 +
[[category:database]]
 +
[[category:NoSQL]]
 +
[[category:java]]
 +
[[category:apache]]
 +
[[category:facebook]]
 +
[[category:recommender system]]
 +
[[category:huihoo]]

2021年8月20日 (五) 05:17的最后版本

Wikipedia-35x35.png 您可以在Wikipedia上了解到此条目的英文信息 Apache Cassandra Thanks, Wikipedia.

Apache Cassandra是一套开源分布式Key-Value存储系统。它最初由Facebook开发，用于储存特别大的数据。Facebook目前在使用此系统。

Cassandra.png
Datastax.png

目录

[编辑] 理论基础

[编辑] 主要特性

  • 分布式
  • 基于Column的结构化
  • 高度可伸展性

2 nodes can handle 100,000 transactions per second, 4 nodes will support 200,000 transactions/sec and 8 nodes will tackle 400,000 transactions/sec。

Cassandra-supplies-linear-scalability.png

Cassandra的主要特点就是它不是一个数据库，而是由一堆数据库节点共同构成的一个分布式网络服务，对Cassandra 的一个写操作，会被复制到其他节点上去，对Cassandra的读操作，也会被路由到某个节点上面去读取。对于一个Cassandra群集来说，扩展性能是比较简单的事情，只管在群集里面添加节点就可以了。

Cassandra是一个混合型的非关系型数据库，类似于Google的BigTable。其主要功能比 Dynomite（分布式的Key-Value存储系统）更丰富，但支持度却不如文档存储MongoDB（介于关系数据库和非关系数据库之间的开源产品，是非关系数据库当中功能最丰富，最像关系型数据库。支持的数据结构非常松散，是类似JSON的bjson格式，因此可以存储比较复杂的数据类型）Cassandra最初由Facebook开发，后转变成了开源项目。它是一个网络社交云计算方面理想的数据库。以Amazon专有的完全分布式的Dynamo为基础，结合了Google BigTable基于列族（Column Family）的数据模型。P2P去中心化的存储。很多方面都可以称之为Dynamo 2.0。

和其他数据库比较，有几个突出特点：

  • 模式灵活：使用Cassandra，像文档存储，你不必提前解决记录中的字段。你可以在系统运行时随意的添加或移除字段。这是一个惊人的效率提升，特别是在大型部署上。
  • 真正的可扩展性：Cassandra是纯粹意义上的水平扩展。为给集群添加更多容量，可以指向另一台电脑。你不必重启任何进程，改变应用查询，或手动迁移任何数据。
  • 多数据中心识别：你可以调整你的节点布局来避免某一个数据中心起火，一个备用的数据中心将至少有每条记录的完全复制。

一些使Cassandra提高竞争力的其他功能：

  • 范围查询：如果你不喜欢全部的键值查询，则可以设置键的范围来查询。
  • 列表数据结构：在混合模式可以将超级列添加到5维。对于每个用户的索引，这是非常方便的。
  • 分布式写操作：有可以在任何地方任何时间集中读或写任何数据。并且不会有任何单点失败。

[编辑] 版本

What’s New in Cassandra 2.2: JSON Support

  • 1.x

[编辑] 指南

[编辑] OS X

brew install cassandra 
brew info cassandra
cassandra -f

bin/cassandra -f
bin/cqlsh
or cqlsh 1.2.3.4 9042 // ip, port
cqlsh> help
cqlsh> describe keyspaces;
cqlsh> use system;
cqlsh:system> select * from schema_keyspaces; // 所有keyspaces
cqlsh:system> describe schema_keyspaces; // schema_keyspaces所有表和表定义
cqlsh> CREATE KEYSPACE mykeyspace
   ... WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
cqlsh> use mykeyspace;
cqlsh:mykeyspace> CREATE TABLE users (
              ... user_id int PRIMARY KEY,
              ... fname text,
              ... lname text
              ... );
cqlsh:mykeyspace> INSERT INTO users (user_id,  fname, lname)
              ... VALUES (1745, 'john', 'smith');
cqlsh:mykeyspace> INSERT INTO users (user_id,  fname, lname)
              ... VALUES (1744, 'john', 'doe');
cqlsh:mykeyspace> INSERT INTO users (user_id,  fname, lname)
              ... VALUES (1746, 'john', 'smith');
cqlsh:mykeyspace> select * from users;

 user_id | fname | lname
---------+-------+-------
    1745 |  john | smith
    1744 |  john |   doe
    1746 |  john | smith

cqlsh:mykeyspace> select now(), uuid(), token() from mykeyspace.users;

You have to be logged in and not anonymous to perform this request 

vim cassandra.yaml 
authenticator: PasswordAuthenticator

创建新用户 

cassandra@cqlsh> create role huihoo with superuser = true and login = true and password = 'huihoo';
cassandra@cqlsh> list roles;

[编辑] Tools

cassandra-stress tool 

cd apache-cassandra-2.1.8/tools/bin
cassandra-stress help -schema
cassandra-stress write n=1000000 // 写100万行
cassandra-stress read n=200000 // 读20万行
cassandra-stress write n=1000000 cl=one -mode native cql3 -schema keyspace="stress" -log file=~/load_1M_rows.log // 写入100万行
cqlsh:mykeyspace> select count(*) from stress.standard1;
 count
---------
 1000000
cqlsh> describe stress.standard1;
cqlsh> select * from stress.standard1 limit 10;

[编辑] 系统信息

nodetool --host 127.0.0.1 cfstats
cqlsh> describe system;
cqlsh> select * from system.batchlog;
cqlsh> select * from system.compaction_history;
cqlsh> select * from system.compactions_in_progress;
cqlsh> select * from system.hints;
cqlsh> select * from system.local;
cqlsh> select * from system.peers;
cqlsh> select * from system.peer_events;
cqlsh> select * from system.range_xfers;
cqlsh> select * from system.sstable_activity;
cqlsh> select * from system.schema_columnfamilies;
cqlsh> select * from system.schema_columns;
cqlsh> select * from system.schema_triggers;
cqlsh> select * from system.schema_usertypes;
cqlsh> select * from system.size_estimates;
cqlsh> select * from system.schema_keyspaces;
cqlsh> select * from mykeyspace.users;
cqlsh> select * from system_traces.sessions;
cqlsh> select * from system_traces.events;

[编辑] CQL

CQL and Java type comparison 

CQL              Java
===              ====
boolean          java.lang.Boolean
int              java.lang.Integer
bigint           java.lang.Long
float            java.lang.Float
double           java.lang.Double
inet             java.net.InetAddress
text             java.lang.String
ascii            java.lang.String
timestamp        java.util.Date
uuid             java.util.UUID
timeuuid         java.util.UUID
varint           java.math.BigInteger
decimal          java.math.BigDecimal
blob             java.nio.ByteBuffer
list<E>          java.util.List<E>      where E is also a type from this list
set<E>           java.util.Set<E>       where E is also a type from this list
map<K,V>         java.util.Map<K,V>     where K and V is also a types from this list
(user type)      com.datastax.driver.core.UDTValue
(tuple type)     com.datastax.driver.core.TupleValue

[编辑] 项目

Apache Cassandra Topics

[编辑] C++

ScyllaDB 是用 C++ 重写的 Apache Cassandra，完全兼容 Cassandra.

[编辑] .NET

[编辑] Python

sudo pip install ipython-cql
sudo pip install virtualenvwrapper
source /usr/local/bin/virtualenvwrapper.sh
git clone https://github.com/rustyrazorblade/python-presentation
cd python-presentation
mkvirtualenv tutorial
pip install -r requirements.txt
ipython notebook

[编辑] MariaDB

[编辑] Docker

[编辑] Spark

[编辑] Hadoop

Hadoop Support

[编辑] Presto

Cassandra Connector

[编辑] 集群HA

[编辑] 控制台

[编辑] IDE

Building and IDE Integration 

$ ./bin/server.sh
http://127.0.0.1:9091
$ pkill -f studio

[编辑] Operator

[编辑] 驱动

[编辑] 厂商

[编辑] 案例

目前，Cassandra对于Netflix而言是首选数据库，因为它们几乎满足了Netflix的所有需求。Netflix已经将95%的数据存储在Cassandra上，包括客户账户信息、影片评分、影片元数据、影片书签和日志等。Netflix在750多个节点上运行着50多个Cassandra集群。高峰时，Netflix每秒要处理50,000多个读取和100,000写入操作。Netflix平均每天要处理21亿次的读取与43亿次的写入操作。

[编辑] 迁移

数据库迁移

[编辑] 文档

[编辑] 图集

  • Column

  • Row

  • Column family

  • Keyspace

  • Super

  • CFS文件系统

  • 虚拟节点

  • Cassandra+Kafka+Akka

  • OpsCenter架构

  • DevCenter

  • 个性化推荐引擎

  • RabbitMQ-Storm-Cassandra实时分析

  • 实时分析集成Esper

  • 微服务平台

  • 写路径

  • 读路径

  • 关系型和Cassandra

  • Cassandra中的表

  • 与关系型对比

  • eBay用户模型

  • eBay用户模型

  • 分布式索引

  • 多数据中心一致性

  • Gossip反熵算法

  • Keyspace

  • Keyspace多数据中心

  • Cassandra+Spark分析数据中心

  • JConsole

  • Gartner魔力象限

  • RDBMS模型

  • Cassandra模型

  • MariaDB存储引擎

  • Cassandra集群

  • Cassandra集群

  • Cassandra集群

  • Cassandra集群

  • K8s运营

[编辑] 链接

来自“http://wiki.huihoo.com/wiki/?title=Apache_Cassandra&oldid=157837”
分享您的观点
个人工具
名字空间

变换
操作
导航
工具箱