欢迎大家赞助一杯啤酒🍺 我们准备了下酒菜:Formal mathematics/Isabelle/ML, Formal verification/Coq/ACL2, C++/F#/Lisp
Apache Mesos
小 (→用户) |
小 (→图集) |
||
(未显示1个用户的41个中间版本) | |||
第4行: | 第4行: | ||
==新闻== | ==新闻== | ||
− | *[https://mesosphere.com/blog/kubernetes-on-mesosphere-dcos-now-certified-by-cncf/ Kubernetes on Mesosphere DC/OS now certified by CNCF] | + | *[https://mesosphere.com/blog/series-d/ Mesosphere Raises $125 Million Series D] (2018.05) |
+ | *[https://mesosphere.com/blog/kubernetes-on-mesosphere-dcos-now-certified-by-cncf/ Kubernetes on Mesosphere DC/OS now certified by CNCF] [https://github.com/mesosphere/dcos-kubernetes-quickstart Kubernetes on DC/OS] | ||
*[https://mesosphere.com/blog/dcos-1_10-kubernetes/ DC/OS 1.10: One Platform for the Containerized Future] | *[https://mesosphere.com/blog/dcos-1_10-kubernetes/ DC/OS 1.10: One Platform for the Containerized Future] | ||
*一些在Twitter和Airbnb中使用Mesos的工程师们一起创建了Mesosphere。(2013.04) | *一些在Twitter和Airbnb中使用Mesos的工程师们一起创建了Mesosphere。(2013.04) | ||
第47行: | 第48行: | ||
==发行版== | ==发行版== | ||
*[https://dcos.io/ Mesosphere DC/OS] | *[https://dcos.io/ Mesosphere DC/OS] | ||
+ | *[[Huihoo on DCOS|Huihoo on DC/OS]] | ||
− | ==DC/ | + | ==DC/OS== |
− | [http://blog.csdn.net/qianggezhishen/article/details/51745702 DC/OS 介绍] | + | *[http://dockone.io/article/1231 开源DC/OS介绍:容器的最佳运行平台] |
+ | *[http://blog.csdn.net/qianggezhishen/article/details/51745702 DC/OS 介绍] | ||
+ | |||
+ | ===版本=== | ||
+ | *[https://docs.mesosphere.com/1.11/ Mesosphere DC/OS 1.11] | ||
+ | *[https://docs.mesosphere.com/1.10/ Mesosphere DC/OS 1.10] | ||
===安装=== | ===安装=== | ||
− | [https://github.com/dcos/dcos-vagrant 本地安装DC/OS集群] | + | 可参考 |
+ | *[https://gist.github.com/cantbewong/38a4f5dc8c78b17c9ca8881e00310498 Install DC/OS on bare metal] | ||
+ | *[http://wiselyman.iteye.com/blog/2200267 下一代云计算平台Apache Mesos定制自己的PaaS-1(Apache Mesos安装部署)] | ||
+ | |||
+ | [https://github.com/dcos/dcos-vagrant 本地安装DC/OS集群],可参考[https://oliverveits.wordpress.com/2017/04/15/getting-started-with-dcos-on-vagrant/ Getting Started with DC/OS on Vagrant] | ||
vagrant plugin install vagrant-hostmanager | vagrant plugin install vagrant-hostmanager | ||
git clone https://github.com/dcos/dcos-vagrant | git clone https://github.com/dcos/dcos-vagrant | ||
第105行: | 第116行: | ||
Unable to login to your DC/OS cluster. Clusters must be connected to the internet. | Unable to login to your DC/OS cluster. Clusters must be connected to the internet. | ||
+ | |||
+ | install_prereqs has been already executed on this host, exiting... | ||
+ | sudo rm -rf dcos-genconf.2d45a8f9e277a60007-5c14d12c3c8eecebcd.tar genconf/ | ||
+ | sudo bash dcos_generate_config.sh --web -v | ||
+ | |||
+ | Docker is configured with a production storage driver, Docker存储问题 | ||
+ | # systemctl stop docker | ||
+ | # rm -rf /var/lib/docker | ||
+ | vi /etc/sysconfig/docker | ||
+ | OPTIONS='--selinux-enabled=false' | ||
+ | vi /etc/sysconfig/docker-storage | ||
+ | DOCKER_STORAGE_OPTIONS= -s overlay | ||
+ | # systemctl start docker | ||
+ | |||
+ | Checking if port 53 (required by spartan) is in use: FAIL | ||
+ | # netstat -ntlp 53 | ||
+ | tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 1567/dnsmasq | ||
+ | |||
+ | [https://serverfault.com/questions/727998/dnsmasq-on-virbr0-libvirt-and-network-manager dnsmasq on virbr0 (libvirt) and network manager] | ||
+ | virsh net-destroy default | ||
+ | virsh net-undefine default | ||
+ | |||
+ | Error executing DC/OS components health check: exit status 127 | ||
+ | |||
+ | 使用systemctl status去看相关服务的状态: | ||
+ | dcos-metrics-master.service // activating (auto-restart) | ||
+ | dcos-metrics-master.socket // active (listening) | ||
+ | dcos-metrics-agent.service // activating (start-pre) | ||
+ | dcos-metrics-agent.socket // active (listening) | ||
+ | $ ntptime | ||
+ | $ sudo service ntpd restart | ||
+ | $ ntptime | ||
+ | $ systemctl status dcos-navstar.service | ||
+ | |||
+ | /opt/mesosphere/bin/pkgpanda uninstall && rm -fr /opt/mesosphere | ||
+ | |||
+ | 将所有节点重启,再 sudo bash dcos_generate_config.sh --postflight | ||
+ | |||
+ | 或Uninstall 所有节点 /opt/mesosphere/bin/pkgpanda uninstall && rm -fr /opt/mesosphere 再重新以GUI的方式安装。 | ||
+ | |||
+ | 并试下[https://github.com/dcos/dcos-diagnostics dcos-diagnostics]工具 | ||
+ | |||
+ | ===卸载=== | ||
+ | [http://www.cnblogs.com/xiashiwendao/p/7859825.html 卸载DC/OS] | ||
===Examples=== | ===Examples=== | ||
第124行: | 第179行: | ||
$ dcos package repo add local-universe http://m1.dcos:8082/repo | $ dcos package repo add local-universe http://m1.dcos:8082/repo | ||
... | ... | ||
+ | |||
+ | ===文档=== | ||
+ | [http://docs.huihoo.com/apache/mesos/1.2.0/design-docs/ Mesos设计文档] | ||
+ | |||
+ | DC/OS官方文档 | ||
+ | |||
+ | 构建[https://github.com/mesosphere/dcos-docs-site/wiki/Local-Build DC/OS文档] 在build/和build-swagger/目录下会生成各个版本的文档,需要[[Node.js]] 8.1.2版本。 | ||
+ | git clone https://github.com/mesosphere/dcos-docs-site | ||
+ | cd dcos-docs-site | ||
+ | npm install | ||
+ | make build-api | ||
+ | npm run dev | ||
+ | localhost:3000 | ||
+ | |||
+ | ===常见问题=== | ||
+ | 可能出现的问题:[https://github.com/sass/node-sass/issues/1527 Error: Missing binding] | ||
+ | $ npm rebuild node-sass | ||
+ | 修改文档路径:webpack.config.js | ||
+ | path: __dirname + './build', | ||
+ | 构建时缺少css, js和assets文件,从docs.mesosphere.com拷贝这些文件,并建立一个新的文档镜像:http://dcos.huihoo.com 和 http://dcos.huihoo.com/swagger | ||
+ | |||
+ | [http://www.yangqiang.im/?p=672 DC/OS 无法获取PUBLIC_IP] | ||
==项目== | ==项目== | ||
− | [[Mesos ecosystem]] | + | [[Mesos ecosystem]] [[文件:Awesome.png]] |
*[https://github.com/dcos/dcos-ui DC/OS UI] | *[https://github.com/dcos/dcos-ui DC/OS UI] | ||
*[https://github.com/dcos/dcos-docker DC/OS Docker] | *[https://github.com/dcos/dcos-docker DC/OS Docker] | ||
第140行: | 第217行: | ||
==用户== | ==用户== | ||
− | * | + | [https://github.com/apache/mesos/blob/master/docs/powered-by-mesos.md Apache Mesos 用户] |
+ | *Apple、Cisco、eBay、PayPal、Netflix、Twitter、Uber、Airbnb、[https://github.com/douban 豆瓣]微博、小米、今日头条 | ||
*Apple 的 Siri 平台运行Mesos之上,拥有几千个节点。(2015.04) | *Apple 的 Siri 平台运行Mesos之上,拥有几千个节点。(2015.04) | ||
*[http://www.infoq.com/cn/news/2017/02/scaling-twitter-infrastructure Twitter的支撑架构:扩展网络与存储并提供服务] | *[http://www.infoq.com/cn/news/2017/02/scaling-twitter-infrastructure Twitter的支撑架构:扩展网络与存储并提供服务] | ||
第152行: | 第230行: | ||
*[http://www.infoq.com/cn/news/2015/10/force12io-microscaling-mesos Force12.io为Apache Mesos创建“微伸缩”框架] | *[http://www.infoq.com/cn/news/2015/10/force12io-microscaling-mesos Force12.io为Apache Mesos创建“微伸缩”框架] | ||
*Verizon 成功演示了在90秒内使用Mesos和Mesosphere的Marathon框架启动 50,000 个容器。(2015.08) | *Verizon 成功演示了在90秒内使用Mesos和Mesosphere的Marathon框架启动 50,000 个容器。(2015.08) | ||
− | |||
==图集== | ==图集== | ||
第158行: | 第235行: | ||
image:mesos-the-heart-of-the-data-center-operating-system.png|数据中心操作系统的内核 | image:mesos-the-heart-of-the-data-center-operating-system.png|数据中心操作系统的内核 | ||
image:mesos-cluster.png|Mesos集群 | image:mesos-cluster.png|Mesos集群 | ||
− | image:apache-mesos-architecture.jpg| | + | image:mesos-cluster-private-public-agent.png|Mesos集群内/外网 |
+ | image:dcos-architecture-layers.png|DC/OS架构 | ||
+ | image:apache-mesos-architecture.jpg|Mesos架构 | ||
image:mesos-two-level-scheduler.png|两层调度 | image:mesos-two-level-scheduler.png|两层调度 | ||
image:mesosphere.png|Mesosphere | image:mesosphere.png|Mesosphere | ||
image:apache-mesos-architecture-example.jpg|架构示例 | image:apache-mesos-architecture-example.jpg|架构示例 | ||
image:DCOS-Dasboard.png|DC/OS仪表盘 | image:DCOS-Dasboard.png|DC/OS仪表盘 | ||
+ | image:dcos-dashboard.jpeg|DC/OS仪表盘 | ||
image:DCOS-Practical-Architecture.png|DC/OS参考架构 | image:DCOS-Practical-Architecture.png|DC/OS参考架构 | ||
image:Mesosphere-DCOS-Mesos.png|Mesosphere DC/OS | image:Mesosphere-DCOS-Mesos.png|Mesosphere DC/OS | ||
image:Mesosphere-DCOS.png|Mesosphere DC/OS | image:Mesosphere-DCOS.png|Mesosphere DC/OS | ||
image:dcos-enterprise-components.png|DC/OS组件 | image:dcos-enterprise-components.png|DC/OS组件 | ||
+ | image:dcos-metrics.png|DC/OS Metrics | ||
+ | image:dcos-metrics-architecture.png|DC/OS Metrics架构 | ||
+ | image:dcos-architecture-distributed-process-management-seq-diagram.png|分布式处理管理 | ||
image:mesos-dns.png|Mesos-DNS | image:mesos-dns.png|Mesos-DNS | ||
image:apache-mesos-test-cluster.png|集群测试 | image:apache-mesos-test-cluster.png|集群测试 | ||
第178行: | 第261行: | ||
image:Mesos-Internals-02.png|Mesos Internals | image:Mesos-Internals-02.png|Mesos Internals | ||
image:mosos-in-qunar.png|去哪儿网 | image:mosos-in-qunar.png|去哪儿网 | ||
+ | image:install-kubernetes-on-dcos.png|Kubernetes安装 | ||
image:DCOS-GUI-Installation.png|DC/OS GUI安装 | image:DCOS-GUI-Installation.png|DC/OS GUI安装 | ||
image:DCOS-GUI-Installation-Preflight.png|DC/OS GUI安装过程 | image:DCOS-GUI-Installation-Preflight.png|DC/OS GUI安装过程 | ||
image:Exhibitor-for-ZooKeeper.png|ZooKeeper监视器 | image:Exhibitor-for-ZooKeeper.png|ZooKeeper监视器 | ||
image:aliyun-docker-repo.png|阿里Docker镜像加速器 | image:aliyun-docker-repo.png|阿里Docker镜像加速器 | ||
+ | image:mesos-docker-cicd.png|CI/CD研发部署 | ||
+ | image:5servers-running-dcos-cloud.jpeg|5服务器的DC/OS云 | ||
</gallery> | </gallery> | ||
第192行: | 第278行: | ||
*[https://jira.mesosphere.com Mesosphere Jira] | *[https://jira.mesosphere.com Mesosphere Jira] | ||
*[http://docs.huihoo.com/apache/mesos Apache Mesos文档] | *[http://docs.huihoo.com/apache/mesos Apache Mesos文档] | ||
+ | *[http://dcos.huihoo.com/ Mesosphere DC/OS 文档镜像] | ||
+ | *[http://docs.huihoo.com/apache/mesos/chrisrc.me/ DC/OS修炼之路] | ||
*[http://www.infoq.com/cn/master-mesos 深入浅出Mesos] | *[http://www.infoq.com/cn/master-mesos 深入浅出Mesos] | ||
*[http://open.mesosphere.com/intro-course/ Advanced Mesos Course,一个非常好的教材,推荐] | *[http://open.mesosphere.com/intro-course/ Advanced Mesos Course,一个非常好的教材,推荐] |
2022年4月29日 (五) 04:44的最后版本
您可以在Wikipedia上了解到此条目的英文信息 Apache Mesos Thanks, Wikipedia. |
Apache Mesos 是一个分布式、集群管理系统和框架,被称为是分布式系统的内核。
目录 |
[编辑] 新闻
- Mesosphere Raises $125 Million Series D (2018.05)
- Kubernetes on Mesosphere DC/OS now certified by CNCF Kubernetes on DC/OS
- DC/OS 1.10: One Platform for the Containerized Future
- 一些在Twitter和Airbnb中使用Mesos的工程师们一起创建了Mesosphere。(2013.04)
- 2010年12月,Mesos 进入了Apache孵化项目。2013年6月,Mesos成功从Apache孵化项目毕业,成为顶级项目。
[编辑] 简介
Mesos是分布式操作系统内核,Mesos是以与Linux内核同样的原则而创建的,不同点仅仅是在于抽象的层面。Mesos内核运行在每一个机器上,同时通过 API 为各种应用提供跨数据中心和云的资源管理调度能力。这些应用包括Hadoop、Spark、Kafka、Elasticsearch。还可配合框架 Marathon 来管理大规模的Docker等容器化应用。
Mesos特性:
- 可扩展到10000个节点;
- 使用 ZooKeeper 实现 Master 和 Slave 的容错;
- 支持 Docker 容器;
- 使用 Linux 容器实现本地任务隔离;
- 多资源调度能力(内存,CPU、磁盘、端口);
- 提供 Java,Python,C++等多种语言 APIs;
- 通过 Web 界面查看集群状态;
- 新版本将支持更多功能。
[编辑] 生态
[编辑] 指南
OS X
brew install mesos
[编辑] 演示
安装一个 mesos-demo 虚拟机
$ wget http://downloads.mesosphere.io/demo/mesos.box // 注: 此文件有2.8G $ vagrant box add --name mesos-demo mesos.box $ git clone https://github.com/mesosphere/RENDLER.git $ cd RENDLER $ vagrant up http://10.141.141.10:5050/ // Mesos http://10.141.141.10:8080 // Marathon $ vagrant ssh $ cd hostfiles/python $ python rendler.py http://mesosphere.io 127.0.1.1:5050 42 <Ctrl+C> to stop... $ cd hostfiles $ bin/make-pdf
[编辑] 发行版
[编辑] DC/OS
[编辑] 版本
[编辑] 安装
可参考
本地安装DC/OS集群,可参考Getting Started with DC/OS on Vagrant
vagrant plugin install vagrant-hostmanager git clone https://github.com/dcos/dcos-vagrant cd dcos-vagrant cp VagrantConfig-1m-1a-1p.yaml VagrantConfig.yaml vagrant up 或 vagrant up --debug
/etc/hosts
192.168.65.90 m1.dcos 192.168.65.111 a1.dcos 192.168.65.60 p1.dcos 192.168.65.50 boot.dcos ssh [email protected] http://m1.dcos/ ci/dcos-install-cli.sh
常见问题,boot超时,调整config.vm.boot_timeout数值,修改Vagrantfile文件。
Vagrant.configure("2") do |config| config.vm.boot_timeout = 600
An error occurred in the underlying SSH library that Vagrant uses.The error message is shown below. In many cases, errors from this library are caused by ssh-agent issues. Try disabling your SSH agent or removing some keys and try again.
升级VirtualBox到5.1.x以上,然后
vagrant destroy vagrant up
VirtualBox: mount.vboxsf: mounting failed with the error: No such device
vagrant plugin install vagrant-vbguest vagrant vbguest
[boot] GuestAdditions versions on your host (5.1.30) and guest (5.1.18) do not match.
vagrant plugin install vagrant-vbguest 或 vagrant vbguest --do install vagrant box update
访问master IPs的Private SSH Key在/Users/huihoo/.vagrant.d/insecure_private_key 或 dcos-vagrant/.vagrant/dcos/private_key_vagrant
cp <path-to-key> genconf/ssh_key && chmod 0600 genconf/ssh_key
运行 sudo bash dcos_generate_config.sh --postflight 出现的错误:detect_ip returned stderr: Device "eth0" does not exist
CentOS 7.x系统中网卡命名规则被重新定义,可能会是”enp0s8”等,CentOS下找不到eth0设备的解决方法。
cd /etc/sysconfig/network-scripts/ sudo touch ifcfg-eth0 样本 NAME=eth0 DEVICE=eth0 IPADDR=192.168.65.111 NETMASK=255.255.255.0 NETWORK=192.168.65.0 GATEWAY=192.168.65.1 BROADCAST=192.168.65.255 ONBOOT=yes USERCTL=yes BOOTPROTO=static HWADDR=08:00:27:c1:f9:cc
Unable to login to your DC/OS cluster. Clusters must be connected to the internet.
install_prereqs has been already executed on this host, exiting...
sudo rm -rf dcos-genconf.2d45a8f9e277a60007-5c14d12c3c8eecebcd.tar genconf/ sudo bash dcos_generate_config.sh --web -v
Docker is configured with a production storage driver, Docker存储问题
# systemctl stop docker # rm -rf /var/lib/docker vi /etc/sysconfig/docker OPTIONS='--selinux-enabled=false' vi /etc/sysconfig/docker-storage DOCKER_STORAGE_OPTIONS= -s overlay # systemctl start docker
Checking if port 53 (required by spartan) is in use: FAIL
# netstat -ntlp 53 tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 1567/dnsmasq
dnsmasq on virbr0 (libvirt) and network manager
virsh net-destroy default virsh net-undefine default
Error executing DC/OS components health check: exit status 127
使用systemctl status去看相关服务的状态:
dcos-metrics-master.service // activating (auto-restart) dcos-metrics-master.socket // active (listening) dcos-metrics-agent.service // activating (start-pre) dcos-metrics-agent.socket // active (listening) $ ntptime $ sudo service ntpd restart $ ntptime $ systemctl status dcos-navstar.service
/opt/mesosphere/bin/pkgpanda uninstall && rm -fr /opt/mesosphere
将所有节点重启,再 sudo bash dcos_generate_config.sh --postflight
或Uninstall 所有节点 /opt/mesosphere/bin/pkgpanda uninstall && rm -fr /opt/mesosphere 再重新以GUI的方式安装。
并试下dcos-diagnostics工具
[编辑] 卸载
[编辑] Examples
[编辑] 命令行
$ dcos marathon app list $ docker images $ docker run mesosphere/mesos_exporter -master http://localhost:5050 $ docker run mesosphere/mesos_exporter -slave http://localhost:5051
[编辑] Universe
安装、使用Mesosphere Universe包仓库。
$ wget https://downloads.mesosphere.com/universe/public/local-universe.tar.gz // 有9.1G $ docker load < local-universe.tar.gz ... $ dcos package repo add local-universe http://m1.dcos:8082/repo ...
[编辑] 文档
DC/OS官方文档
构建DC/OS文档 在build/和build-swagger/目录下会生成各个版本的文档,需要Node.js 8.1.2版本。
git clone https://github.com/mesosphere/dcos-docs-site cd dcos-docs-site npm install make build-api npm run dev localhost:3000
[编辑] 常见问题
可能出现的问题:Error: Missing binding
$ npm rebuild node-sass
修改文档路径:webpack.config.js
path: __dirname + './build',
构建时缺少css, js和assets文件,从docs.mesosphere.com拷贝这些文件,并建立一个新的文档镜像:http://dcos.huihoo.com 和 http://dcos.huihoo.com/swagger
[编辑] 项目
- DC/OS UI
- DC/OS Docker
- Mantl
- Ochopod + Mesos + Marathon + CLI == PaaS
- Vamp - The Very Awesome Microservices Platform, written in Scala, with some parts in Go. Vamp package
[编辑] 文档
- Mesos,数据中心操作系统的核心
- Apache Mesos as an SDK for Building Distributed Frameworks
- Apache Mesos + Apache YARN = Myriad
- Mesos: Elastically Scalable Operations, Simplified.
[编辑] 用户
- Apple、Cisco、eBay、PayPal、Netflix、Twitter、Uber、Airbnb、豆瓣微博、小米、今日头条
- Apple 的 Siri 平台运行Mesos之上,拥有几千个节点。(2015.04)
- Twitter的支撑架构:扩展网络与存储并提供服务
- 从HashiCorp Nomad对上百万容器进行调度所学到的经验
- Netflix使用Apache Mesos打造分布式资源调度系统
- Mesos在去哪儿网的实践之路,随着业务线日志的逐步接入,日志平台已经增长成为单日处理60亿条日志/6TB数据的庞大平台。
- Autodesk如何使用Mesos来实现自己的PaaS云平台 Autodesk基于Mesos的可扩展事件系统 Autodesk is Forging Ahead with Mesos, Containers and DC/OS
- 知乎万级规模容器平台架构和实战
- 中国移动浙江公司数据中心操作系统(DCOS)实践
- 从太平洋保险DCOS实践看传统企业的魅力变革
- Force12.io为Apache Mesos创建“微伸缩”框架
- Verizon 成功演示了在90秒内使用Mesos和Mesosphere的Marathon框架启动 50,000 个容器。(2015.08)
[编辑] 图集
[编辑] 链接
- Apache Mesos官网
- DC/OS官网
- Apache Mesos @ GitHub
- DC/OS @ GitHub
- DC/OS Labs @ GitHub
- Mesosphere Jira
- Apache Mesos文档
- Mesosphere DC/OS 文档镜像
- DC/OS修炼之路
- 深入浅出Mesos
- Advanced Mesos Course,一个非常好的教材,推荐
- 深入解析DC/OS 1.8 – 高可靠的微服务及大数据管理平台 Mesos源码分析
- Open source datacenter computing with Apache Mesos
- 从闭源的Borg到开源的Mesos
- Mesos与OpenStack,谁才是私有云的未来?