Add Support for Pod Affinity/Anti-Affinity
Currently with MaestroV2, our pods with replicas are being deployed on the same nodes at times which defeats the purpose of our highly available pods running on a single node in our multi-node cluster.
I'm proposing adding a feature to Maestro to incorporate the Kubernetes concept of Pod Affinity/Anti-Affinity to allow Kubernetes to schedule pods to separate nodes based on what pods are already scheduled on the nodes.
Here is the documentation from Kubernetes on the Pod Affinity/Anti-Affinity concept: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity
Kasia Hoffman commented