Access Kubernetes Objects Data From /Proc Directory
An overview of the “proc” directory and some use cases
Overview of /proc directory
The /proc directory is an amazing concept. It doesn’t really exist, yet we can analyze it. Its zero-length files are neither binary nor text, yet we can examine and display them. This special directory holds all the details about our Linux system, such as — kernel, processes, and configuration parameters. By understanding the /proc directory, we can learn how Linux commands works, and we can even do some administrative tasks.
The /proc directory is organized in virtual directories and subdirectories, and it groups files by similar topic. Working as a root, the ls /proc the command brings up something like this:
>> cd /proc
>> ls
1 19 21220 24840 28408 30323 34754 477 614 87 consoles kmsg self
10 190 21254 24908 28466 30324 34755 478 621 88 cpuinfo kpagecgroup slabinfo
100 192 21259 24947 285 30325 34756 488 70 89 crypto kpagecount softirqs
109 194 21292 24991 28847 30326 34757 490 701 9 devices kpageflags stat
11 196 214 25051 28850 30327 34758 493 71 90 diskstats loadavg swaps
112 199 216 25081 28881 30328 34759 544 72 91 dma locks sys
12 2 21980 25735 28949 32163 34760 547 7204 92 driver mdstat sysrq-trigger
125 20 22 25763 28955 32257 34761 548 73 93 execdomains meminfo sysvipc
13406 200 22553 25781 28957 32280 34762 559 74 94 fb misc thread-self
14 202 22878 25827 29175 32288 34763 560 75 95 filesystems modules timer_list
15 205 23 25847 29933 32554 34786 561 76 96 fs mounts tty
16 207 23840 25848 29961 32598 34810 567 77 97 interrupts mtrr uptime
164 21 23936 25966 3 34153 355 569 78 98 iomem net version
165 210 23993 26046 30202 34748 390 570 8 99 ioports pagetypeinfo version_signature
166 211 24 26071 30239 34749 4 5788 81 acpi irq partitions vmallocinfo
17 21123 24037 28222 30319 34750 400 588 82 buddyinfo kallsyms pressure vmstat
18 21135 244 28284 30320 34751 474 595 84 bus kcore sched_debug zoneinfo
186 21137 24751 28355 30321 34752 475 6 85 cgroups key-users schedstat
188 21208 24800 284 30322 34753 476 606 86 cmdline keys scsiMore about [ Proc ]
Access kubernetes object data from /proc directory
Suppose, we are in a kubeadm based on Kubernetes Cluster. And we have access to the controlplane (master node) where our kubernetes components are running. If our etcd (key-value store) is running as a pod inside the controlplane node, we can easily access data stored on etcd by leveraging the/proc directory. For that, we have to login into the controlplane node. And then we need to find out the process ID of the etcd.
Find out the process ID of the etcd:
# Find out the etcd process id
controlplane $ ps aux | grep -i etcd
root 25966 2.5 2.2 11214776 46124 ? Ssl 04:52 0:10 etcd --advertise-client-urls=https://172.30.1.2:2379 root 28957 4.6 15.4 1112260 314108 ? Ssl 04:53 0:14 kube-apiserver --advertise-address=172.30.1.2 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/etc/kubernetes/pki/ca.crt --enable-admission-plugins=NodeRestriction --enable-bootstrap-token-auth=true --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/etc/kubernetes/pki/sa.pub --service-account-signing-key-file=/etc/kubernetes/pki/sa.key --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/etc/kubernetes/pki/apiserver.crt --tls-private-key-file=/etc/kubernetes/pki/apiserver.keyBy executing “ps aux” command we have extracted the process id of the etcd. Now, move to the “/proc/25966” directory and check the contents of that directory :
controlplane $ cd /proc/25966
controlplane $ ls -la
...
lrwxrwxrwx 1 root root 0 Dec 14 05:21 exe -> /usr/local/bin/etcd
-r-------- 1 root root 0 Dec 14 05:21 environ
dr-x------ 2 root root 0 Dec 14 05:13 fd
...Under the “/proc/25966” folder there are lots of files and folders. But currently, we are interested in “exe”, “environ” and “fd” files.
exe — Symlink to the process’s binary.
fd— Contains all file descriptors for a process, showing which files or devices it is using.
environ — Shows all environment variables for the process.
Access kubernetes “secrets” data:
Now, let’s create a secret object on the kubernetes cluster :
controlplane $ kubectl create secret generic test-secret \
--from-literal=secret-password=53546235476523645And then move to the “/proc/etcd-process-ID/fd” directory and list all files:
controlplane $ cd /proc/25966/fd
controlplane $ ls -la
...
lrwx------ 1 root root 64 Dec 14 05:31 10 -> /var/lib/etcd/member/snap/db #***
...
lrwx------ 1 root root 64 Dec 14 05:31 14 -> 'socket:[83836]'
lrwx------ 1 root root 64 Dec 14 05:31 15 -> 'socket:[83939]'
lrwx------ 1 root root 64 Dec 14 05:31 16 -> 'socket:[83940]'There are lots of files under the “/proc/etcd-process-ID/fd” directory. But we are interested in a file named “/var/lib/etcd/member/snap/db” which contains all the etcd data. In our case file number is “10” (This may differ in your case).
Now, inspect the file number “10” and try to access kubernetes secrets :
[Secret-key]
controlplane $ cat 10 | strings | grep -A2 secret-password
6{"f:data":{".":{},"f:secret-password":{}},"f:type":{}}B
secret-password #<---
53546235476523645 #<---
Opaque
--
6{"f:data":{".":{},"f:secret-password":{}},"f:type":{}}B
secret-password #<---
53546235476523645 #<---
OpaqueAs we can see from the above illustration, we were able to get Kubernetes secrets data from /proc directory.
Access pods “environment” variables:
Create a new pod with an environment variable defined:
controlplane $ kubectl run pod --image=httpd --env="USERNAME=admin"
controlplane $ kubectl get pod pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
pod 1/1 Running 0 25s 192.168.1.3 node01 #*<-- As we can see that our pod is running on the worker-node name node01. So, the process running for the pod resides in the “node01” worker node. Let’s ssh into the “node01” worker node. And then try to find out the process-id of the pod which is running a “httpd” container.
node01 $ ssh node01
node01 $ ps aux | grep -i httpd
root 37792 0.2 0.2 5996 4688 ? Ss 06:09 0:00 httpd -DFOREGROUND
...As we are running “httpd” container inside the pod. We have searched the process running for “httpd” service.
Now, let’s inspect the “/proc/httpd_process_id/environ” file:
node01 $ cat /proc/37792/environ | strings | grep -i USERNAME
USERNAME=adminAs we can see from the above illustration, from /proc directory we were able to access env variables of the pod which is running a “httpd” container.
