Start your Flink cluster with docker

A Flink cluster consists of 2 components JobManager: Accepts jobs from users. Allocate tasks of job to TaskManager. TaskManager: Executes the tasks.

We will use docker to launch 2 containers. 1 for Jobmanager and 1 for Taskmanager.

cd ~/CUHK-CSE-Lab-Flink
docker-compose up

Now, you should see all the standard output and error output from the containers. Don't touch the terminal. Open a new terminal and ssh to node-1 again

ssh node-1

In the following section, we need to access Flink WebUI. We need to know the ports that they are listening to.

Execute the following command to get the port of Flink WebUI

docker port <your cse user name>_jobmanager | grep -o ':[0-9]\+'