欢迎大家赞助一杯啤酒🍺 我们准备了下酒菜:Formal mathematics/Isabelle/ML, Formal verification/Coq/ACL2, C++/F#/Lisp
Apache Cassandra
小 (→链接) |
小 (→项目) |
||
(未显示1个用户的107个中间版本) | |||
第1行: | 第1行: | ||
− | |||
{{SeeWikipedia}} | {{SeeWikipedia}} | ||
Apache Cassandra是一套开源分布式Key-Value存储系统。它最初由[[Facebook]]开发,用于储存特别大的数据。Facebook目前在使用此系统。 | Apache Cassandra是一套开源分布式Key-Value存储系统。它最初由[[Facebook]]开发,用于储存特别大的数据。Facebook目前在使用此系统。 | ||
+ | [[文件:cassandra.png|right]] | ||
+ | [[文件:datastax.png|right]] | ||
+ | |||
+ | ==理论基础== | ||
+ | *peer-to-peer、环形架构基于 Amazon [[Dynamo]] | ||
+ | *数据存储模型基于 Google [[BigTable]] | ||
+ | *[http://www.planetcassandra.org/blog/cassandra-daughter-of-dynamo-and-bigtable/ Cassandra: Daughter of Dynamo and BigTable] | ||
+ | *[[gossip protocol]] | ||
+ | *[[Staged event-driven architecture|SEDA]] | ||
==主要特性== | ==主要特性== | ||
第25行: | 第33行: | ||
*列表数据结构:在混合模式可以将超级列添加到5维。对于每个用户的索引,这是非常方便的。 | *列表数据结构:在混合模式可以将超级列添加到5维。对于每个用户的索引,这是非常方便的。 | ||
*分布式写操作:有可以在任何地方任何时间集中读或写任何数据。并且不会有任何单点失败。 | *分布式写操作:有可以在任何地方任何时间集中读或写任何数据。并且不会有任何单点失败。 | ||
+ | |||
+ | ==版本== | ||
+ | *[http://docs.huihoo.com/apache/cassandra/4.x/doc/latest/ 4.x] | ||
+ | *3.x:[http://docs.huihoo.com/apache/cassandra/3.9/html/ 3.9], [http://docs.huihoo.com/javadoc/apache/cassandra/3.9/ 3.9 javadoc] | ||
+ | *2.x | ||
+ | [http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-2-json-support What’s New in Cassandra 2.2: JSON Support] | ||
+ | *1.x | ||
==指南== | ==指南== | ||
− | OS X | + | ===OS X=== |
brew install cassandra | brew install cassandra | ||
+ | brew info cassandra | ||
cassandra -f | cassandra -f | ||
bin/cassandra -f | bin/cassandra -f | ||
bin/cqlsh | bin/cqlsh | ||
+ | or cqlsh 1.2.3.4 9042 // ip, port | ||
cqlsh> help | cqlsh> help | ||
− | + | cqlsh> describe keyspaces; | |
+ | cqlsh> use system; | ||
+ | cqlsh:system> select * from schema_keyspaces; // 所有keyspaces | ||
+ | cqlsh:system> describe schema_keyspaces; // schema_keyspaces所有表和表定义 | ||
cqlsh> CREATE KEYSPACE mykeyspace | cqlsh> CREATE KEYSPACE mykeyspace | ||
... WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 }; | ... WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 }; | ||
第57行: | 第77行: | ||
1746 | john | smith | 1746 | john | smith | ||
− | (3 | + | cqlsh:mykeyspace> select now(), uuid(), token() from mykeyspace.users; |
+ | |||
+ | You have to be logged in and not anonymous to perform this request | ||
+ | vim cassandra.yaml | ||
+ | authenticator: PasswordAuthenticator | ||
+ | 创建新用户 | ||
+ | cassandra@cqlsh> create role huihoo with superuser = true and login = true and password = 'huihoo'; | ||
+ | cassandra@cqlsh> list roles; | ||
+ | |||
+ | ===Tools=== | ||
+ | [http://docs.datastax.com/en/cassandra/2.2/cassandra/tools/toolsCStress.html cassandra-stress tool] | ||
+ | cd apache-cassandra-2.1.8/tools/bin | ||
+ | cassandra-stress help -schema | ||
+ | cassandra-stress write n=1000000 // 写100万行 | ||
+ | cassandra-stress read n=200000 // 读20万行 | ||
+ | cassandra-stress write n=1000000 cl=one -mode native cql3 -schema keyspace="stress" -log file=~/load_1M_rows.log // 写入100万行 | ||
+ | cqlsh:mykeyspace> select count(*) from stress.standard1; | ||
+ | count | ||
+ | --------- | ||
+ | 1000000 | ||
+ | cqlsh> describe stress.standard1; | ||
+ | cqlsh> select * from stress.standard1 limit 10; | ||
+ | |||
+ | ===系统信息=== | ||
+ | nodetool --host 127.0.0.1 cfstats | ||
+ | cqlsh> describe system; | ||
+ | cqlsh> select * from system.batchlog; | ||
+ | cqlsh> select * from system.compaction_history; | ||
+ | cqlsh> select * from system.compactions_in_progress; | ||
+ | cqlsh> select * from system.hints; | ||
+ | cqlsh> select * from system.local; | ||
+ | cqlsh> select * from system.peers; | ||
+ | cqlsh> select * from system.peer_events; | ||
+ | cqlsh> select * from system.range_xfers; | ||
+ | cqlsh> select * from system.sstable_activity; | ||
+ | cqlsh> select * from system.schema_columnfamilies; | ||
+ | cqlsh> select * from system.schema_columns; | ||
+ | cqlsh> select * from system.schema_triggers; | ||
+ | cqlsh> select * from system.schema_usertypes; | ||
+ | cqlsh> select * from system.size_estimates; | ||
+ | cqlsh> select * from system.schema_keyspaces; | ||
+ | cqlsh> select * from mykeyspace.users; | ||
+ | cqlsh> select * from system_traces.sessions; | ||
+ | cqlsh> select * from system_traces.events; | ||
+ | |||
+ | ==CQL== | ||
+ | *[https://cassandra.apache.org/doc/cql3/ Cassandra Query Language (CQL) v3] | ||
+ | *[http://docs.huihoo.com/apache/cassandra/planetcassandra/CQL-This-is-not-he-SQL-you-are-looking-for.pdf CQL: This is not the SQL you are looking for] | ||
+ | *[http://docs.huihoo.com/apache/cassandra/datastax/CQL-3.1-for-Cassandra-2.0-and-2.1.pdf CQL for Cassandra 2.x] | ||
+ | |||
+ | [http://www.planetcassandra.org/blog/user-defined-functions-in-cassandra-3-0/ CQL and Java type comparison] | ||
+ | CQL Java | ||
+ | === ==== | ||
+ | boolean java.lang.Boolean | ||
+ | int java.lang.Integer | ||
+ | bigint java.lang.Long | ||
+ | float java.lang.Float | ||
+ | double java.lang.Double | ||
+ | inet java.net.InetAddress | ||
+ | text java.lang.String | ||
+ | ascii java.lang.String | ||
+ | timestamp java.util.Date | ||
+ | uuid java.util.UUID | ||
+ | timeuuid java.util.UUID | ||
+ | varint java.math.BigInteger | ||
+ | decimal java.math.BigDecimal | ||
+ | blob java.nio.ByteBuffer | ||
+ | list<E> java.util.List<E> where E is also a type from this list | ||
+ | set<E> java.util.Set<E> where E is also a type from this list | ||
+ | map<K,V> java.util.Map<K,V> where K and V is also a types from this list | ||
+ | (user type) com.datastax.driver.core.UDTValue | ||
+ | (tuple type) com.datastax.driver.core.TupleValue | ||
+ | |||
+ | ==项目== | ||
+ | [https://github.com/topics/cassandra Apache Cassandra Topics] | ||
+ | *[https://github.com/strapdata/elassandra Elassandra] = [[Elasticsearch]] + Apache Cassandra | ||
+ | *[https://github.com/OpenNMS/newts Newts] is a time-series data store based on Apache Cassandra. | ||
+ | |||
+ | ==C++== | ||
+ | [[ScyllaDB]] 是用 [[C++]] 重写的 Apache Cassandra,完全兼容 Cassandra. | ||
+ | |||
+ | ==.NET== | ||
+ | *[https://github.com/datastax/csharp-driver DataStax C# Driver for Apache Cassandra] | ||
==Python== | ==Python== | ||
− | [https://github.com/rustyrazorblade/python-presentation Python & Cassandra - Best Friends] | + | *[https://github.com/twissandra/twissandra Twissandra] |
+ | *[https://github.com/rustyrazorblade/python-presentation Python & Cassandra - Best Friends] | ||
sudo pip install ipython-cql | sudo pip install ipython-cql | ||
sudo pip install virtualenvwrapper | sudo pip install virtualenvwrapper | ||
第69行: | 第172行: | ||
pip install -r requirements.txt | pip install -r requirements.txt | ||
ipython notebook | ipython notebook | ||
+ | |||
+ | ==MariaDB== | ||
+ | *[http://www.infoq.com/cn/news/2012/11/Cassandra-SE MariaDB的Cassandra存储引擎] 允许[[MariaDB]]通过标准SQL语法使用Cassandra集群。 | ||
+ | *[https://mariadb.com/kb/en/mariadb/cassandra/ Cassandra Storage Engine] | ||
+ | |||
+ | ==Docker== | ||
+ | *[http://docs.huihoo.com/datastax/Best-Practices-for-Running-DataStax-Enterprise-within-Docker.pdf Best Practices for Running DataStax Enterprise within Docker] | ||
+ | |||
+ | ==[[Apache Spark|Spark]]== | ||
+ | *[http://www.datastax.com/dev/blog/zen-art-spark-maintenance Zen and the Art of Spark Maintenance] | ||
+ | *[https://github.com/datastax/spark-cassandra-connector DataStax Spark Cassandra Connector] | ||
+ | *[https://tobert.github.io/post/2014-07-15-installing-cassandra-spark-stack.html Installing the Cassandra / Spark OSS Stack] | ||
+ | |||
+ | ==[[Apache Hadoop|Hadoop]]== | ||
+ | [https://wiki.apache.org/cassandra/HadoopSupport Hadoop Support] | ||
+ | |||
+ | ==[[Presto]]== | ||
+ | [https://prestodb.io/docs/current/connector/cassandra.html Cassandra Connector] | ||
+ | |||
+ | ==集群HA== | ||
+ | *[https://github.com/riptano/ccm CCM (Cassandra Cluster Manager)] | ||
+ | *[https://github.com/mesosphere/cassandra-mesos Cassandra on Mesos] | ||
+ | |||
+ | ==控制台== | ||
+ | *[https://github.com/Netflix/Priam Priam (Netflix)] | ||
+ | *[https://github.com/suguru/cassandra-webconsole cassandra-webconsole] | ||
+ | *[https://github.com/sebgiroux/Cassandra-Cluster-Admin Cassandra Cluster Admin] | ||
+ | *[https://github.com/hmsonline/virgil Virgil is a services layer for Cassandra] | ||
+ | *[https://github.com/tbarbugli/cassandra_snapshotter Cassandra Snapshotter] | ||
+ | *[https://github.com/hailocab/ctop CTOP ("Top for Cassandra")] | ||
+ | |||
+ | ==IDE== | ||
+ | [https://cassandra.apache.org/doc/latest/development/ide.html Building and IDE Integration] | ||
+ | *[https://downloads.datastax.com/#devcenter DataStax DevCenter] 已不再更新维护 [https://github.com/Patipat-Chuensuwannakul/devcenter @ GitHub] | ||
+ | *[https://downloads.datastax.com/#desktop DataStax Desktop] | ||
+ | *[https://downloads.datastax.com/#studio DataStax Studio] Studio only supports DataStax Enterprise clusters. | ||
+ | $ ./bin/server.sh | ||
+ | http://127.0.0.1:9091 | ||
+ | $ pkill -f studio | ||
+ | *[https://hackolade.com Hackolade] | ||
+ | |||
+ | ==Operator== | ||
+ | *[https://github.com/instaclustr/cassandra-operator Kubernetes Operator for Cassandra] | ||
+ | |||
+ | ==驱动== | ||
+ | *[http://www.planetcassandra.org/apache-cassandra-client-drivers/ Apache Cassandra Client Drivers] | ||
+ | *[https://github.com/Netflix/astyanax Astyanax] is a high level Java client for Apache Cassandra | ||
+ | *[https://pypi.python.org/pypi/django-cassandra-engine/ django-cassandra-engine] - the Cassandra backend for [[Django]] | ||
+ | *[https://github.com/impetus-opensource/Kundera Kundera] | ||
+ | *pip install cassandra-driver // python | ||
+ | |||
+ | ==厂商== | ||
+ | *[[DataStax]] 下载 [http://www.planetcassandra.org/cassandra/ DataStax Community Edition — Apache Cassandra] | ||
+ | *[http://www.satellitevolta.com/ Satellite Volta] | ||
+ | *[http://www.seastar.io/ Seastar] | ||
==案例== | ==案例== | ||
第76行: | 第234行: | ||
*[http://tech.qq.com/a/20140807/006287.htm 这款数据库曾被Facebook抛弃 现正帮苹果壮大] Apple's, with over 75,000 nodes storing over 10 PB of data. | *[http://tech.qq.com/a/20140807/006287.htm 这款数据库曾被Facebook抛弃 现正帮苹果壮大] Apple's, with over 75,000 nodes storing over 10 PB of data. | ||
*[http://www.planetcassandra.org/blog/post/soundcloud-activity-feed-and-real-time-stats-powered-by-apache-cassandra SoundCloud's Activity Feed and Real-Time Stats Powered by Apache Cassandra] | *[http://www.planetcassandra.org/blog/post/soundcloud-activity-feed-and-real-time-stats-powered-by-apache-cassandra SoundCloud's Activity Feed and Real-Time Stats Powered by Apache Cassandra] | ||
+ | *[http://www.planetcassandra.org/blog/interview/facebooks-instagram-making-the-switch-to-cassandra-from-redis-a-75-insta-savings/ Facebook’s Instagram: Making the Switch to Cassandra from Redis, a 75% ‘Insta’ Savings] | ||
*[http://www.planetcassandra.org/blog/post/nexgate-choses-cassandra-over-mongodb-for-a-multi-master-nosql-solution Nexgate Chooses Cassandra over MongoDB for their Multi-Master NoSQL Solution] | *[http://www.planetcassandra.org/blog/post/nexgate-choses-cassandra-over-mongodb-for-a-multi-master-nosql-solution Nexgate Chooses Cassandra over MongoDB for their Multi-Master NoSQL Solution] | ||
*[http://www.planetcassandra.org/blog/post/urban-airship-utilizing-cassandra-to-connect-hundreds-of-millions-of-devices-for-breaking-news-alerts Urban Airship Utilizing Cassandra to Connect Hundreds of Millions of Devices for Breaking News Alerts] | *[http://www.planetcassandra.org/blog/post/urban-airship-utilizing-cassandra-to-connect-hundreds-of-millions-of-devices-for-breaking-news-alerts Urban Airship Utilizing Cassandra to Connect Hundreds of Millions of Devices for Breaking News Alerts] | ||
第82行: | 第241行: | ||
*[http://www.planetcassandra.org/blog/post/appdynamics-utilizes-cassandra-for-app-monitoring-and-metrics-tracking AppDynamics Utilizes Cassandra for App Monitoring and Metrics Tracking] | *[http://www.planetcassandra.org/blog/post/appdynamics-utilizes-cassandra-for-app-monitoring-and-metrics-tracking AppDynamics Utilizes Cassandra for App Monitoring and Metrics Tracking] | ||
*[https://tech.coursera.org/blog/2014/09/23/courseras-adoption-of-cassandra/ Coursera’s Adoption of Cassandra] | *[https://tech.coursera.org/blog/2014/09/23/courseras-adoption-of-cassandra/ Coursera’s Adoption of Cassandra] | ||
− | *[http://www.infoq.com/cn/articles/spotify-migrate-cassandra Spotify是怎样从Postgres切换至Cassandra的?] | + | *[http://www.infoq.com/cn/articles/spotify-migrate-cassandra Spotify是怎样从Postgres切换至Cassandra的?] [http://www.planetcassandra.org/blog/personalization-at-spotify-using-apache-cassandra/ Personalization at Spotify Using Apache Cassandra], [http://www.planetcassandra.org/blog/interview/spotify-scales-to-the-top-of-the-charts-with-apache-cassandra-at-40k-requestssecond/ Spotify scales to the top of the charts with Apache Cassandra at 40k requests/second] |
+ | *[http://www.planetcassandra.org/blog/interview/fighting-fraud-at-nosql-scale-rsa-migrates-from-oracle-to-apache-cassandra-to-protect-your-online-banking/ RSA migrates from Oracle to Apache Cassandra to protect your online banking] | ||
+ | *[http://www.planetcassandra.org/blog/interview/gaming-dev-platform-unity-powers-up-with-cassandra-migrates-away-from-mongodb-for-a-scalable-low-latency-solution/ Gaming dev platform Unity powers up with Cassandra; migrates away from MongoDB for a scalable low latency solution] | ||
+ | *[http://www.planetcassandra.org/blog/interview/coursera-migrates-to-the-top-of-the-class-moves-to-cassandra-for-an-always-on-on-demand-classroom/ Coursera migrates to the top of the class; moves their over 9 million students to Cassandra for an always on, on-demand classroom] | ||
+ | *[http://www.planetcassandra.org/blog/interview/multimedia-messaging-app-cubie-is-ready-to-grow-worry-free-with-apache-cassandra/ Multimedia messaging app Cubie is ready to grow, worry free, with Apache Cassandra] | ||
+ | *[http://www.planetcassandra.org/blog/multi-datacenter-cassandra-on-32-raspberry-pis/ Multi-Datacenter Cassandra on 32 Raspberry Pi’s] | ||
==迁移== | ==迁移== | ||
第95行: | 第259行: | ||
*[http://docs.huihoo.com/apache/spark/summit/east2015/SSE15-11-Delivering-Meaning-At-High-Velocity-with-Spark-Streaming-Cassandra-Kafka-and-Akka.pdf Delivering Meaning In NearReal Time At High Velocity & Massive Scale] | *[http://docs.huihoo.com/apache/spark/summit/east2015/SSE15-11-Delivering-Meaning-At-High-Velocity-with-Spark-Streaming-Cassandra-Kafka-and-Akka.pdf Delivering Meaning In NearReal Time At High Velocity & Massive Scale] | ||
*[http://docs.huihoo.com/oreilly/conferences/strataconf/big-data-conference-ny-2013/An-Introduction-to-Real-Time-Analytics-with-Cassandra-and-Hadoop.pdf An Introduction to Real-Time Analytics with Cassandra and Hadoop] | *[http://docs.huihoo.com/oreilly/conferences/strataconf/big-data-conference-ny-2013/An-Introduction-to-Real-Time-Analytics-with-Cassandra-and-Hadoop.pdf An Introduction to Real-Time Analytics with Cassandra and Hadoop] | ||
+ | *[http://docs.huihoo.com/apache/cassandra/Cassandra-Data-Modeling-Best-Practices-at-eBay.pdf Cassandra Data Modeling Best Practices at eBay] [http://docs.huihoo.com/apache/cassandra/Cassandra-at-eBay.pdf Cassandra at eBay] [http://docs.huihoo.com/apache/cassandra/planetcassandra/Cassandra-at-eBay-Scale.pdf Cassandra Scale at eBay] | ||
==图集== | ==图集== | ||
<gallery> | <gallery> | ||
+ | image:cassandra-column.png|Column | ||
+ | image:cassandra-row.png|Row | ||
+ | image:cassandra-column-family.png|Column family | ||
+ | image:cassandra-keyspace.png|Keyspace | ||
+ | image:cassandra-super-column-and-super-column-family.png|Super | ||
+ | image:Cassandra-File-System-CFS.png|CFS文件系统 | ||
+ | image:cassandra-virtual-nodes.png|虚拟节点 | ||
image:Spark-Streaming-Cassandra-Kafka-and-Akka.png|Cassandra+Kafka+Akka | image:Spark-Streaming-Cassandra-Kafka-and-Akka.png|Cassandra+Kafka+Akka | ||
image:opscenter-architecture.png|OpsCenter架构 | image:opscenter-architecture.png|OpsCenter架构 | ||
+ | image:Datastax-DevCenter.png|DevCenter | ||
image:recommendation-engine-personalization.png|个性化推荐引擎 | image:recommendation-engine-personalization.png|个性化推荐引擎 | ||
+ | image:RabbitMQ-Storm-Cassandra.png|RabbitMQ-Storm-Cassandra实时分析 | ||
+ | image:RabbitMQ-Storm-Esper-Cassandra.png|实时分析集成Esper | ||
+ | image:Microservice-platform.png|微服务平台 | ||
image:cassandra-write-path.png|写路径 | image:cassandra-write-path.png|写路径 | ||
image:cassandra-read-path.png|读路径 | image:cassandra-read-path.png|读路径 | ||
image:traditional-and-cassandra-data-modeling.png|关系型和Cassandra | image:traditional-and-cassandra-data-modeling.png|关系型和Cassandra | ||
image:tables-in-cassandra.png|Cassandra中的表 | image:tables-in-cassandra.png|Cassandra中的表 | ||
+ | image:relational-model-vs-cassandra-model.png|与关系型对比 | ||
+ | image:ebay-user-cassandra-model.png|eBay用户模型 | ||
+ | image:ebay-cassandra-data-model.png|eBay用户模型 | ||
+ | image:cassandra-distributed-Index.png|分布式索引 | ||
+ | image:Cassandra-multiple-DC-Consistency-LocalOne.png|多数据中心一致性 | ||
+ | image:cassandra-anti-entropy-repair.png|Gossip反熵算法 | ||
+ | image:cassandra-keyspace-02.png|Keyspace | ||
+ | image:cassandra-keyspace-02-multi-dc.png|Keyspace多数据中心 | ||
+ | image:cassandra-and-spark-analysis-data-center.png|Cassandra+Spark分析数据中心 | ||
+ | image:jconsole-cassandra.png|JConsole | ||
+ | image:Gartner-Magic-Quadrant-for-Operational-Database-Management-Systems-October-2015.png|Gartner魔力象限 | ||
+ | image:A-simple-hotel-search-system-using-RDBMS.png|RDBMS模型 | ||
+ | image:The-hotel-search-represented-with-Cassandra-model.png|Cassandra模型 | ||
+ | image:mariadb-cassandra-storage-engine.png|MariaDB存储引擎 | ||
+ | image:32-node-raspberry-pi-cassandra-cluster-01.jpg|Cassandra集群 | ||
+ | image:32-node-raspberry-pi-cassandra-cluster-04.jpg|Cassandra集群 | ||
+ | image:32-node-raspberry-pi-cassandra-cluster-02.jpg|Cassandra集群 | ||
+ | image:32-node-raspberry-pi-cassandra-cluster-03.jpg|Cassandra集群 | ||
+ | image:Cassandra-Kubernetes-Operator.jpg|K8s运营 | ||
</gallery> | </gallery> | ||
==链接== | ==链接== | ||
*[http://cassandra.apache.org/ Apache Cassandra官方网站] | *[http://cassandra.apache.org/ Apache Cassandra官方网站] | ||
− | *[ | + | *[https://github.com/apache/cassandra Cassandra @ GitHub] |
+ | *[https://www.datastax.com/blog DataStax Blog] | ||
+ | *[http://docs.huihoo.com/apache/cassandra/ Apache Cassandra文档] | ||
+ | *[http://docs.datastax.com/en/ DataStax Cassandra文档] | ||
*[https://www.ibm.com/developerworks/cn/opensource/os-apache-cassandra/ 考虑 Apache Cassandra 数据库] | *[https://www.ibm.com/developerworks/cn/opensource/os-apache-cassandra/ 考虑 Apache Cassandra 数据库] | ||
*[http://www.infoq.com/cn/articles/best-practice-of-cassandra-data-model-design Cassandra数据模型设计最佳实践(上部)] | *[http://www.infoq.com/cn/articles/best-practice-of-cassandra-data-model-design Cassandra数据模型设计最佳实践(上部)] | ||
*[http://www.infoq.com/cn/articles/best-practices-cassandra-data-model-design-part2 Cassandra数据模型设计最佳实践(下部)] | *[http://www.infoq.com/cn/articles/best-practices-cassandra-data-model-design-part2 Cassandra数据模型设计最佳实践(下部)] | ||
+ | *[http://www.planetcassandra.org/blog/the-most-important-thing-to-know-in-cassandra-data-modeling-the-primary-key/ The most important thing to know in Cassandra data modeling: The primary key] | ||
+ | *[http://www.infoq.com/cn/articles/cassandra-2nd-edition-book-review 《Cassandra权威指南》第二版书评及访谈] | ||
− | + | [[category:database]] | |
− | + | ||
[[category:NoSQL]] | [[category:NoSQL]] | ||
[[category:java]] | [[category:java]] | ||
[[category:apache]] | [[category:apache]] | ||
[[category:facebook]] | [[category:facebook]] | ||
+ | [[category:recommender system]] | ||
+ | [[category:huihoo]] |
2021年8月20日 (五) 05:17的最后版本
您可以在Wikipedia上了解到此条目的英文信息 Apache Cassandra Thanks, Wikipedia. |
Apache Cassandra是一套开源分布式Key-Value存储系统。它最初由Facebook开发,用于储存特别大的数据。Facebook目前在使用此系统。
目录 |
[编辑] 理论基础
- peer-to-peer、环形架构基于 Amazon Dynamo
- 数据存储模型基于 Google BigTable
- Cassandra: Daughter of Dynamo and BigTable
- gossip protocol
- SEDA
[编辑] 主要特性
- 分布式
- 基于Column的结构化
- 高度可伸展性
2 nodes can handle 100,000 transactions per second, 4 nodes will support 200,000 transactions/sec and 8 nodes will tackle 400,000 transactions/sec。
Cassandra的主要特点就是它不是一个数据库,而是由一堆数据库节点共同构成的一个分布式网络服务,对Cassandra 的一个写操作,会被复制到其他节点上去,对Cassandra的读操作,也会被路由到某个节点上面去读取。对于一个Cassandra群集来说,扩展性能是比较简单的事情,只管在群集里面添加节点就可以了。
Cassandra是一个混合型的非关系型数据库,类似于Google的BigTable。其主要功能比 Dynomite(分布式的Key-Value存储系统)更丰富,但支持度却不如文档存储MongoDB(介于关系数据库和非关系数据库之间的开源产品,是非关系数据库当中功能最丰富,最像关系型数据库。支持的数据结构非常松散,是类似JSON的bjson格式,因此可以存储比较复杂的数据类型)Cassandra最初由Facebook开发,后转变成了开源项目。它是一个网络社交云计算方面理想的数据库。以Amazon专有的完全分布式的Dynamo为基础,结合了Google BigTable基于列族(Column Family)的数据模型。P2P去中心化的存储。很多方面都可以称之为Dynamo 2.0。
和其他数据库比较,有几个突出特点:
- 模式灵活:使用Cassandra,像文档存储,你不必提前解决记录中的字段。你可以在系统运行时随意的添加或移除字段。这是一个惊人的效率提升,特别是在大型部署上。
- 真正的可扩展性:Cassandra是纯粹意义上的水平扩展。为给集群添加更多容量,可以指向另一台电脑。你不必重启任何进程,改变应用查询,或手动迁移任何数据。
- 多数据中心识别:你可以调整你的节点布局来避免某一个数据中心起火,一个备用的数据中心将至少有每条记录的完全复制。
一些使Cassandra提高竞争力的其他功能:
- 范围查询:如果你不喜欢全部的键值查询,则可以设置键的范围来查询。
- 列表数据结构:在混合模式可以将超级列添加到5维。对于每个用户的索引,这是非常方便的。
- 分布式写操作:有可以在任何地方任何时间集中读或写任何数据。并且不会有任何单点失败。
[编辑] 版本
- 4.x
- 3.x:3.9, 3.9 javadoc
- 2.x
What’s New in Cassandra 2.2: JSON Support
- 1.x
[编辑] 指南
[编辑] OS X
brew install cassandra brew info cassandra cassandra -f
bin/cassandra -f bin/cqlsh or cqlsh 1.2.3.4 9042 // ip, port cqlsh> help cqlsh> describe keyspaces; cqlsh> use system; cqlsh:system> select * from schema_keyspaces; // 所有keyspaces cqlsh:system> describe schema_keyspaces; // schema_keyspaces所有表和表定义 cqlsh> CREATE KEYSPACE mykeyspace ... WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 }; cqlsh> use mykeyspace; cqlsh:mykeyspace> CREATE TABLE users ( ... user_id int PRIMARY KEY, ... fname text, ... lname text ... ); cqlsh:mykeyspace> INSERT INTO users (user_id, fname, lname) ... VALUES (1745, 'john', 'smith'); cqlsh:mykeyspace> INSERT INTO users (user_id, fname, lname) ... VALUES (1744, 'john', 'doe'); cqlsh:mykeyspace> INSERT INTO users (user_id, fname, lname) ... VALUES (1746, 'john', 'smith'); cqlsh:mykeyspace> select * from users;
user_id | fname | lname ---------+-------+------- 1745 | john | smith 1744 | john | doe 1746 | john | smith
cqlsh:mykeyspace> select now(), uuid(), token() from mykeyspace.users;
You have to be logged in and not anonymous to perform this request
vim cassandra.yaml authenticator: PasswordAuthenticator
创建新用户
cassandra@cqlsh> create role huihoo with superuser = true and login = true and password = 'huihoo'; cassandra@cqlsh> list roles;
[编辑] Tools
cd apache-cassandra-2.1.8/tools/bin cassandra-stress help -schema cassandra-stress write n=1000000 // 写100万行 cassandra-stress read n=200000 // 读20万行 cassandra-stress write n=1000000 cl=one -mode native cql3 -schema keyspace="stress" -log file=~/load_1M_rows.log // 写入100万行 cqlsh:mykeyspace> select count(*) from stress.standard1; count --------- 1000000 cqlsh> describe stress.standard1; cqlsh> select * from stress.standard1 limit 10;
[编辑] 系统信息
nodetool --host 127.0.0.1 cfstats cqlsh> describe system; cqlsh> select * from system.batchlog; cqlsh> select * from system.compaction_history; cqlsh> select * from system.compactions_in_progress; cqlsh> select * from system.hints; cqlsh> select * from system.local; cqlsh> select * from system.peers; cqlsh> select * from system.peer_events; cqlsh> select * from system.range_xfers; cqlsh> select * from system.sstable_activity; cqlsh> select * from system.schema_columnfamilies; cqlsh> select * from system.schema_columns; cqlsh> select * from system.schema_triggers; cqlsh> select * from system.schema_usertypes; cqlsh> select * from system.size_estimates; cqlsh> select * from system.schema_keyspaces; cqlsh> select * from mykeyspace.users; cqlsh> select * from system_traces.sessions; cqlsh> select * from system_traces.events;
[编辑] CQL
- Cassandra Query Language (CQL) v3
- CQL: This is not the SQL you are looking for
- CQL for Cassandra 2.x
CQL Java === ==== boolean java.lang.Boolean int java.lang.Integer bigint java.lang.Long float java.lang.Float double java.lang.Double inet java.net.InetAddress text java.lang.String ascii java.lang.String timestamp java.util.Date uuid java.util.UUID timeuuid java.util.UUID varint java.math.BigInteger decimal java.math.BigDecimal blob java.nio.ByteBuffer list<E> java.util.List<E> where E is also a type from this list set<E> java.util.Set<E> where E is also a type from this list map<K,V> java.util.Map<K,V> where K and V is also a types from this list (user type) com.datastax.driver.core.UDTValue (tuple type) com.datastax.driver.core.TupleValue
[编辑] 项目
- Elassandra = Elasticsearch + Apache Cassandra
- Newts is a time-series data store based on Apache Cassandra.
[编辑] C++
ScyllaDB 是用 C++ 重写的 Apache Cassandra,完全兼容 Cassandra.
[编辑] .NET
[编辑] Python
sudo pip install ipython-cql sudo pip install virtualenvwrapper source /usr/local/bin/virtualenvwrapper.sh git clone https://github.com/rustyrazorblade/python-presentation cd python-presentation mkvirtualenv tutorial pip install -r requirements.txt ipython notebook
[编辑] MariaDB
- MariaDB的Cassandra存储引擎 允许MariaDB通过标准SQL语法使用Cassandra集群。
- Cassandra Storage Engine
[编辑] Docker
[编辑] Spark
- Zen and the Art of Spark Maintenance
- DataStax Spark Cassandra Connector
- Installing the Cassandra / Spark OSS Stack
[编辑] Hadoop
[编辑] Presto
[编辑] 集群HA
[编辑] 控制台
- Priam (Netflix)
- cassandra-webconsole
- Cassandra Cluster Admin
- Virgil is a services layer for Cassandra
- Cassandra Snapshotter
- CTOP ("Top for Cassandra")
[编辑] IDE
- DataStax DevCenter 已不再更新维护 @ GitHub
- DataStax Desktop
- DataStax Studio Studio only supports DataStax Enterprise clusters.
$ ./bin/server.sh http://127.0.0.1:9091 $ pkill -f studio
[编辑] Operator
[编辑] 驱动
- Apache Cassandra Client Drivers
- Astyanax is a high level Java client for Apache Cassandra
- django-cassandra-engine - the Cassandra backend for Django
- Kundera
- pip install cassandra-driver // python
[编辑] 厂商
[编辑] 案例
目前,Cassandra对于Netflix而言是首选数据库,因为它们几乎满足了Netflix的所有需求。Netflix已经将95%的数据存储在Cassandra上,包括客户账户信息、影片评分、影片元数据、影片书签和日志等。Netflix在750多个节点上运行着50多个Cassandra集群。高峰时,Netflix每秒要处理50,000多个读取和100,000写入操作。Netflix平均每天要处理21亿次的读取与43亿次的写入操作。
- DataStax也为其他各种行业建立了不同版本的Cassandra工具。DataStax已经筹资8400万美元,目前有员工300多人,正准备IPO。埃利斯称,他们已经有500多家客户,包括“财富100强”中的25家大公司。
- 这款数据库曾被Facebook抛弃 现正帮苹果壮大 Apple's, with over 75,000 nodes storing over 10 PB of data.
- SoundCloud's Activity Feed and Real-Time Stats Powered by Apache Cassandra
- Facebook’s Instagram: Making the Switch to Cassandra from Redis, a 75% ‘Insta’ Savings
- Nexgate Chooses Cassandra over MongoDB for their Multi-Master NoSQL Solution
- Urban Airship Utilizing Cassandra to Connect Hundreds of Millions of Devices for Breaking News Alerts
- Apache Cassandra Powers Yakaz for 10 Million Unique Visitors Every Month
- i2O Water Switches to Cassandra from Microsoft SQL Server, Saving Over 100 Million Liters of Water per Day Across the World
- AppDynamics Utilizes Cassandra for App Monitoring and Metrics Tracking
- Coursera’s Adoption of Cassandra
- Spotify是怎样从Postgres切换至Cassandra的? Personalization at Spotify Using Apache Cassandra, Spotify scales to the top of the charts with Apache Cassandra at 40k requests/second
- RSA migrates from Oracle to Apache Cassandra to protect your online banking
- Gaming dev platform Unity powers up with Cassandra; migrates away from MongoDB for a scalable low latency solution
- Coursera migrates to the top of the class; moves their over 9 million students to Cassandra for an always on, on-demand classroom
- Multimedia messaging app Cubie is ready to grow, worry free, with Apache Cassandra
- Multi-Datacenter Cassandra on 32 Raspberry Pi’s
[编辑] 迁移
数据库迁移
- MySQL to Cassandra Migrations
- Oracle to Cassandra Migrations
- MongoDB to Cassandra Migrations
- HBase to Cassandra Migrations
- Redis to Cassandra Migrations
[编辑] 文档
- Delivering Meaning In NearReal Time At High Velocity & Massive Scale
- An Introduction to Real-Time Analytics with Cassandra and Hadoop
- Cassandra Data Modeling Best Practices at eBay Cassandra at eBay Cassandra Scale at eBay