DC/OS offers its users the ability to install various frameworks with a single click from a central publicly hosted package repository called Universe.
Many Mesosphere clients run their DC/OS clusters in air-gapped environments with no access to the internet, thus not allowing for said one click installation of framework packages.
You can easily solve this problem if your DC/OS cluster is running in such a restricted environment.
The simplest way to give your users access to the packages in the Universe repository is by compiling a copy of it (called local Universe) and hosting it internally to your cluster.
The Mesosphere documentation covers all steps needed to utilize such a copy.
Here we will focus on a more user friendly way of generating the local Universe tarball. The original instructions suggest you should be replacing strings in a Makefile and then running the build. While this works pretty well if you know exactly what is needed, it is not very user friendly.
I have written a bit of automation that will spin up a Vagrant machine or an AWS instance, install all prerequisites needed and present a TUI dialog allowing you to pick and choose all framework packages. Once compiled, the local Universe tarball will be available to download from a web server running on your Vagrant machine or AWS instance.
Here is how to use it on your machine:
git clone https://github.com/dobriak/dcos-universe-builder.git
cd dcos-universe-builder
Vagrant
If you would like to quickly try out building a local Universe, Vagrant is the quickest (and free) way to do it. Please note that the default CentOS/7 Vagrant box used has limited disk size and if you are to include many framework packages, you might run out of space.
vagrant up
Wait for the prerequisites to install, then ssh into the Vagrant machine and run the framework selection script.
vagrant ssh
# In your Vagrant machine
/vagrant/scripts/select_frameworks.sh
AWS instance
Launch an AWS EC2 instance using the Quick Start RHEL 7 AMI, any instance type that can run Docker will work (I use m4.xlarge), assign at least 20GB of disk space. For simplicity, let the instance get a public IP. Select a security group (or create one) that allows external access to port TCP/9999.
Edit aws.config setting the path to your AWS key file, the public IP of your instance and the default user for that AMI (ec2-user for the Quick Start RHEL 7 AMI).
# On your machine
./aws_bootstrap.sh
Once the prerequisites are installed, the script will log you into your AWS EC2 instance, where you can run the framework selection script:
# In your AWS EC2 instance
./select_frameworks.sh
Operating the TUI is very simple, use the arrows and the space bar to select/deselect frameworks. Once happy with your selection, hit the Tab key to switch to the buttons and select “OK”.
The script will ask you if you would like to build the local Universe and then start a webserver , so you can download the (potentially very big) tarball.
# On your machine
curl -O http://127.0.0.1:9999/local-universe.tar.gz
curl -O http://127.0.0.1:9999/local-universe.tar.gz.sha256
# Make sure your file is identical
sha256sum local-universe.tar.gz
cat local-universe.tar.gz.sha256
It is important to make sure that the file you just downloaded is identical to the one generated by the script, as the file can get very big (up to 10-15GB).
To skip all confirmation dialogs you can pass a “-y” parameter to select_frameworks.sh