I made this page to show the gradual progression of a yaml file to show examples of a network policy that becomes more specialized over time. This is to avoid weighing the main article down with lots of repeated code examples. The following procedures should be applicable to ingress network policy as well.
Building a network policy to control egress traffic
Let's start with a yaml file that blocks all egress traffic for pods in the mtlong2-test namespace.
- Block all egress:
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: mtlong2-test-deny-egress
namespace: mtlong2-test
spec:
podSelector: {}policyTypes:
- Egress
---
- We can see that this policy is being enforced by looking at the logs on the weave pod running on the source host. Example:
- We have a pod named 'nginx-deployment-966857787-mwpl6' with an IP address '10.32.0.20' running on kub019. We also have a weave pod running on kub019 called 'weave-net-pbtn8'. I get this result when trying to retrieve a website from inside my nginx pod:
root@nginx-deployment-966857787-mwpl6:/# curl google.com
curl: (6) Could not resolve host: google.com
[mtlong2@lsst-kub017 mtlong2-test]$ kubectl -n kube-system logs weave-net-pbtn8 weave-npc | tail -n 1
WARN: 2018/08/28 19:30:00.279536 UDP connection from 10.32.0.20:43765 to 10.44.0.24:53 blocked by Weave NPC.
As you can see from the above example, one of the first things I noticed when deploying the network policy is that absolutely all outgoing connections are denied. This includes DNS queries to the k8s DNS pods inside the cluster. (the destination IP address 10.44.0.24 above is referring to one of the kube-system DNS pods.) Let's correct that by allowing TCP/UDP traffic on port 53:
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: mtlong2-test-deny-egress
namespace: mtlong2-test
spec:
podSelector: {}policyTypes:
- Egress
egress:
- ports:
- port: 53
protocol: UDP
- port: 53
protocol: TCP
---
Now that we have port 53 open, let's try to talk to Google again:
root@nginx-deployment-966857787-mwpl6:/# curl google.com
curl: (7) Failed to connect to google.com port 80: Connection timed out
[mtlong2@lsst-kub017 mtlong2-test]$ kubectl -n kube-system logs weave-net-pbtn8 weave-npc | tail -n 1
WARN: 2018/08/28 19:53:51.559701 TCP connection from 10.32.0.20:58838 to 172.217.6.110:80 blocked by Weave NPC.
- Judging from the logs, we're now able to resolve DNS queries, but we still can't retrieve the website. Mission accomplished! But what if we only want egress to a port on certain networks? This next policy update will allow DNS queries to the kube-system DNS pods, but not on external name servers like Google (8.8.8.8).
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: mtlong2-test-deny-egress
namespace: mtlong2-test
spec:
podSelector: {}policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 10.0.0.0/8
ports:
- port: 53
protocol: UDP
- port: 53
protocol: TCP
---
- This new policy will allow outgoing connections to port 53 only on the 10.0.0.0/8 (weave) network. Let's test this out:
root@nginx-deployment-966857787-mwpl6:/# curl google.com
curl: (7) Failed to connect to google.com port 80: Connection timed out
[mtlong2@lsst-kub017 mtlong2-test]$ kubectl -n kube-system logs weave-net-pbtn8 weave-npc | tail -n 1
WARN: 2018/08/28 20:14:04.429491 TCP connection from 10.32.0.20:37124 to 216.58.192.174:80 blocked by Weave NPC.
- Same errors as last time. But what if we change our pod's name server to '8.8.8.8'?
root@nginx-deployment-966857787-mwpl6:/etc# echo 'nameserver 8.8.8.8' > /etc/resolv.conf
root@nginx-deployment-966857787-mwpl6:/etc# curl google.com
curl: (6) Could not resolve host: google.com
[mtlong2@lsst-kub017 mtlong2-test]$ kubectl -n kube-system logs weave-net-pbtn8 weave-npc | tail -n 1
WARN: 2018/08/28 20:18:01.795617 UDP connection from 10.32.0.20:39640 to 8.8.8.8:53 blocked by Weave NPC.
- You can also setup network policies to match by namespace or pod instead of ipblock. For our DNS queries, this is probably safer and more desirable longterm. One thing to keep in mind is that pods will need to have their RBAC permissions setup to be able to see the other namespaces. I will update this document with the procedures when I figure that out.
How Network Polices Look Elsewhere on the System
Network policies are created and logged by the Kubernetes api server. However, like most of the Kubernetes networking, they are applied as iptables rules on the host systems. You can get a view of the actual iptables modifications by looking at the logs for the weave-npc container on a relevant host.
Below is an example what it looks like to add the above egress rule allowing port 53 to the Weave network:
$ kubectl -n kube-system logs weave-net-tb2dj weave-npc
INFO: 2018/09/05 21:19:12.404325 deleting entry 10.32.0.20 from weave-rRCXXq+gn%ip@4%gT(2rzL}#r of b808186e-aad5-11e8-b4a3-3cfdfe5b96d0
INFO: 2018/09/05 21:19:12.404363 deleted entry 10.32.0.20 from weave-rRCXXq+gn%ip@4%gT(2rzL}#r of b808186e-aad5-11e8-b4a3-3cfdfe5b96d0
INFO: 2018/09/05 21:19:12.405887 deleting entry 10.40.0.28 from weave-rRCXXq+gn%ip@4%gT(2rzL}#r of b8070764-aad5-11e8-b4a3-3cfdfe5b96d0
INFO: 2018/09/05 21:19:12.405912 deleted entry 10.40.0.28 from weave-rRCXXq+gn%ip@4%gT(2rzL}#r of b8070764-aad5-11e8-b4a3-3cfdfe5b96d0
INFO: 2018/09/05 21:19:12.407467 adding rule [-p TCP -m set --match-set weave-z@i}[/BP1@VL$D$JdBq03jr1[ src -m set --match-set weave-]2^zFy(_q?M2Q}vavlH0ij{|6 dst --dport 53 -m comment --comment pods: namespace: mtlong2-test, selector: -> namespaces: selector: name=kube-system (egress) -j WEAVE-NPC-EGRESS-ACCEPT] to "WEAVE-NPC-EGRESS-CUSTOM" chainINFO: 2018/09/05 21:19:12.413725 adding rule [-p TCP -m set --match-set weave-z@i}[/BP1@VL$D$JdBq03jr1[ src -m set --match-set weave-]2^zFy(_q?M2Q}vavlH0ij{|6 dst --dport 53 -m comment --comment pods: namespace: mtlong2-test, selector: -> namespaces: selector: name=kube-system (egress) -j RETURN] to "WEAVE-NPC-EGRESS-CUSTOM" chainINFO: 2018/09/05 21:19:12.419274 adding rule [-p UDP -m set --match-set weave-z@i}[/BP1@VL$D$JdBq03jr1[ src -m set --match-set weave-]2^zFy(_q?M2Q}vavlH0ij{|6 dst --dport 53 -m comment --comment pods: namespace: mtlong2-test, selector: -> namespaces: selector: name=kube-system (egress) -j WEAVE-NPC-EGRESS-ACCEPT] to "WEAVE-NPC-EGRESS-CUSTOM" chainINFO: 2018/09/05 21:19:12.422234 adding rule [-p UDP -m set --match-set weave-z@i}[/BP1@VL$D$JdBq03jr1[ src -m set --match-set weave-]2^zFy(_q?M2Q}vavlH0ij{|6 dst --dport 53 -m comment --comment pods: namespace: mtlong2-test, selector: -> namespaces: selector: name=kube-system (egress) -j RETURN] to "WEAVE-NPC-EGRESS-CUSTOM" chainAnd here is the rule from `iptables -L` output:
[root@lsst-kub018 ~]# iptables -L -n | grep 53
WEAVE-NPC-EGRESS-ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 match-set weave-z@i}[/BP1@VL$D$JdBq03jr1[ src match-set weave-]2^zFy(_q?M2Q}vavlH0ij{|6 dst tcp dpt:53 /* pods: namespace: mtlong2-test, selector: -> namespaces: selector: name=kube-system (egress) */RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 match-set weave-z@i}[/BP1@VL$D$JdBq03jr1[ src match-set weave-]2^zFy(_q?M2Q}vavlH0ij{|6 dst tcp dpt:53 /* pods: namespace: mtlong2-test, selector: -> namespaces: selector: name=kube-system (egress) */WEAVE-NPC-EGRESS-ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 match-set weave-z@i}[/BP1@VL$D$JdBq03jr1[ src match-set weave-]2^zFy(_q?M2Q}vavlH0ij{|6 dst udp dpt:53 /* pods: namespace: mtlong2-test, selector: -> namespaces: selector: name=kube-system (egress) */RETURN udp -- 0.0.0.0/0 0.0.0.0/0 match-set weave-z@i}[/BP1@VL$D$JdBq03jr1[ src match-set weave-]2^zFy(_q?M2Q}vavlH0ij{|6 dst udp dpt:53 /* pods: namespace: mtlong2-test, selector: -> namespaces: selector: name=kube-system (egress) */