0x374 Virtualization

1. Scheduler

1.2. slurm

how to use slurm

scontrol

scontrol update nodename=<nodename> state=resume

1.3. YARN

ResourceManager

• keeps the metadata of jobs
• hosts on a different host from HDFS NameNode

NodeManager

• run on each node, co-located with HDFS DataNode
• manage YARN container (resource allocation done by resourcemanager)

Orchestration (Kubernetes)

inspired by borg