[{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/posts/","section":"Blog","summary":"","title":"Blog","type":"posts"},{"content":" If you\u0026rsquo;re building a home media server, Jellyfin is the absolute best no-strings-attached solution. But deploying it on a Kubernetes cluster (K3s) running on an immutable OS like Fedora Silverblue? That brings some massive headaches, especially around networking and file permissions.\nAfter spending way too much time falling down the Calico and iptables rabbit holes, I finally got it running flawlessly using the official Helm chart and K3s\u0026rsquo;s default Flannel network. Here is how to get it working without compromising your host\u0026rsquo;s security.\nThe Fedora Silverblue Networking Gotcha # Out of the box, K3s uses Flannel for internal pod networking. Flannel relies on dynamically created virtual network bridges.\nHere is the problem: Fedora Silverblue is immutable and strictly relies on firewalld (backed by nftables). When K3s spins up these virtual interfaces, firewalld has no idea what they are, shoves them into the restrictive public zone, and ruthlessly drops all your internal pod-to-pod traffic. The result? You\u0026rsquo;ll be staring at a 502 Bad Gateway error because your Traefik Ingress simply can\u0026rsquo;t route traffic to your Jellyfin pod.\nYou might be tempted to just disable firewalld or hack the kernel to support legacy iptables. Don\u0026rsquo;t. You can keep your server fully secure by simply telling firewalld to explicitly trust the subnets K3s uses.\nStep 1: Whitelist K3s in Firewalld # K3s uses 10.42.0.0/16 for Pods and 10.43.0.0/16 for Services. We just need to add these to the trusted zone.\nRun these commands on your host:\nsudo systemctl enable --now firewalld sudo firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 sudo firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 sudo firewall-cmd --reload With the network unblocked, go ahead and install K3s and Helm:\n# Install K3s curl -sfL https://get.k3s.io | sh - # Grab your kubeconfig mkdir -p ~/.kube sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config sudo chown $(id -u):$(id -g) ~/.kube/config # Install Helm curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash Create a dedicated namespace for the deployment:\nkubectl create namespace jellyfin Step 2: Map Your Media (PV \u0026amp; PVC) # Jellyfin needs access to your actual media files on the host machine. I use a hostPath Persistent Volume to pass a local directory straight into the cluster.\nCreate a file named media-storage.yaml:\napiVersion: v1 kind: PersistentVolume metadata: name: jellyfin-movies-pv labels: type: local spec: storageClassName: manual capacity: storage: 500Gi accessModes: - ReadWriteMany hostPath: path: \u0026#34;/home/joeri/Movies/\u0026#34; # Change this to your actual media path --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: jellyfin-movies-pvc namespace: jellyfin spec: storageClassName: manual accessModes: - ReadWriteMany resources: requests: storage: 500Gi Apply it:\nkubectl apply -f media-storage.yaml Step 3: Helm Values \u0026amp; The Empty /media Bug # Before we deploy, we need to fix two major configuration traps.\nFirst: File Permissions. Jellyfin runs as a non-root user. If the container\u0026rsquo;s UID doesn\u0026rsquo;t match the host user that owns your media folder, Jellyfin won\u0026rsquo;t see your files. We fix this by passing your host\u0026rsquo;s UID/GID into a podSecurityContext. (Run id on your host terminal to find yours—it\u0026rsquo;s usually 1000).\nSecond: The Empty /media Folder. The official Helm chart defaults to creating a blank emptyDir at /media. If you try to use generic Helm injection keys to override it, it will fail silently and leave you with an empty directory. The fix is to disable the native media block completely and inject standard Kubernetes volumes and volumeMounts arrays.\nCreate jellyfin-values.yaml:\nimage: repository: jellyfin/jellyfin tag: latest pullPolicy: IfNotPresent # Match the Pod ID to the Host ID so Jellyfin can read your files podSecurityContext: runAsUser: 1000 runAsGroup: 1000 fsGroup: 1000 # Set up Traefik Ingress ingress: enabled: true annotations: kubernetes.io/ingress.class: traefik hosts: - host: jellyfin.local # Point this to your server\u0026#39;s IP in /etc/hosts paths: - path: / pathType: Prefix # Standard configuration PVC persistence: config: enabled: true size: 20Gi storageClass: \u0026#34;local-path\u0026#34; # Crucial: Disable the default empty media mount media: enabled: false # Manually mount our PVC to exactly where we want it volumes: - name: movies-volume persistentVolumeClaim: claimName: jellyfin-movies-pvc volumeMounts: - name: movies-volume mountPath: /media/Movies readOnly: true Step 4: Deploy # With the config sorted out, deploy the official chart:\nhelm repo add jellyfin https://jellyfin.github.io/jellyfin-helm helm repo update helm install jellyfin jellyfin/jellyfin \\ --namespace jellyfin \\ -f jellyfin-values.yaml Run kubectl get pods -n jellyfin and wait for the pod to hit Running.\nOnce it\u0026rsquo;s up, map jellyfin.local to your server\u0026rsquo;s IP in your client machine\u0026rsquo;s /etc/hosts file, and open it in your browser. When you add your media library in the setup wizard, point it to /media/Movies and it will scan everything instantly.\nTroubleshooting Ingress \u0026amp; Upgrades # If you can\u0026rsquo;t reach the web interface, the fastest way to isolate the issue is to bypass your browser and test routing directly from your K3s node:\ncurl -v -H \u0026#34;Host: jellyfin.local\u0026#34; http://localhost 302 Found: The cluster is working perfectly. Your client\u0026rsquo;s DNS/hosts file is wrong. 502 Bad Gateway: Traefik is running, but it\u0026rsquo;s being blocked from talking to the pod. Double-check your firewalld trusted zones from Step 1. Connection refused: Traefik isn\u0026rsquo;t listening. Ensure the svclb-traefik pods are actually running in the kube-system namespace. Applying Changes: If you want to edit your jellyfin-values.yaml later (like adding a new volume for TV shows), Kubernetes won\u0026rsquo;t auto-detect the local file changes. You have to push the update using Helm:\nhelm upgrade jellyfin jellyfin/jellyfin --namespace jellyfin -f jellyfin-values.yaml Helm will calculate the diff and spin up a new pod with the fresh configuration. Happy self-hosting!\n","date":"29 June 2026","externalUrl":null,"permalink":"/jellyfin-k3s-helm/","section":"Blog","summary":"","title":"Deploying Jellyfin on Kubernetes with K3s and Helm","type":"posts"},{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/tags/fedora/","section":"Tags","summary":"","title":"Fedora","type":"tags"},{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/tags/helm/","section":"Tags","summary":"","title":"Helm","type":"tags"},{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/tags/jellyfin/","section":"Tags","summary":"","title":"Jellyfin","type":"tags"},{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/","section":"Joeri JM Smissaert","summary":"","title":"Joeri JM Smissaert","type":"page"},{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/tags/k3s/","section":"Tags","summary":"","title":"K3s","type":"tags"},{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/tags/kubernetes/","section":"Tags","summary":"","title":"Kubernetes","type":"tags"},{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/tags/silverblue/","section":"Tags","summary":"","title":"Silverblue","type":"tags"},{"content":"","date":"29 June 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"","date":"20 June 2026","externalUrl":null,"permalink":"/tags/apple-silicon/","section":"Tags","summary":"","title":"Apple Silicon","type":"tags"},{"content":"","date":"20 June 2026","externalUrl":null,"permalink":"/tags/ingress/","section":"Tags","summary":"","title":"Ingress","type":"tags"},{"content":"","date":"20 June 2026","externalUrl":null,"permalink":"/tags/macos/","section":"Tags","summary":"","title":"MacOS","type":"tags"},{"content":"","date":"20 June 2026","externalUrl":null,"permalink":"/tags/minikube/","section":"Tags","summary":"","title":"Minikube","type":"tags"},{"content":" Why vfkit, and not Docker, Colima, or Rancher Desktop? # macOS cannot run Linux containers natively, so every local Kubernetes option boots a Linux VM under the hood. The trouble is where the cluster network lives and how heavy that VM is.\nWith the popular drivers — Minikube\u0026rsquo;s docker driver, Docker Desktop, Colima, or Rancher Desktop (which runs a Lima/Colima-style VM beneath its UI) — your nodes run inside a single, opaque Linux VM. The pod, Service, and Ingress networks exist on a bridge (docker0 and friends) inside that VM, sitting behind NAT. macOS has no route to that subnet, so a 192.168.x.x Service IP or an Ingress address is simply unreachable from your browser. The standard escape hatch is minikube tunnel or port-forwarding — extra processes that need sudo to bind ports 80/443, hang under load, and behave nothing like a real cluster.\nThese tools also lean toward one large, general-purpose appliance VM bundling a full container runtime and management layer. That is convenient for plain docker run, but it deepens the isolation: now your traffic is NAT\u0026rsquo;d twice (host → appliance VM → nested node container), and bridged, host-routable networking is either unavailable or off by default because it requires root.\nThe vfkit driver takes a different approach. It uses vfkit, a thin wrapper around Apple\u0026rsquo;s native Virtualization.framework, to launch a lightweight VM per node — no Docker Desktop, no nested containers. Paired with the vmnet-shared network, Apple\u0026rsquo;s vmnet framework hands each node a real, host-routable IP on the 192.168.64.x subnet, just like a machine on your LAN. No NAT to tunnel through, no minikube tunnel, no port-forwarding — you point your browser at the Ingress IP and it just works, exactly as it would in production.\nPrerequisites # A Mac Homebrew installed. Terminal access with sudo privileges. Step 1: Clean the Slate # If you have tried and failed to set this up before using Homebrew\u0026rsquo;s networking tools, you must remove them. Homebrew installs these tools without the root permissions required to attach to the macOS kernel, which causes silent network failures.\nDelete any hanging clusters and uninstall the Homebrew networking packages:\nminikube delete brew uninstall vmnet-helper socket_vmnet Step 2: Install the Correct Dependencies # We will use Homebrew to install the hypervisor, but we must use the official installation script for the network helper so it properly installs as root.\n1. Install the vfkit hypervisor:\nbrew install vfkit 2. Install vmnet-helper via the official script:\ncurl -fsSL https://github.com/minikube-machine/vmnet-helper/releases/latest/download/install.sh | bash When prompted by the script — Do you want to install the sudoers rule? (y/n) — type y and hit Enter. This allows Minikube to use the network bridge without constantly asking for your password.\n3. Grant manual permission (if you declined the script prompt):\nvmnet-helper must run as root to create a vmnet interface. To let users in the staff group run it without a password, you must install the default sudoers rule. If you declined the automatic prompt in the step above, run this command manually:\nsudo install -m 0640 /opt/vmnet-helper/share/doc/vmnet-helper/sudoers.d/vmnet-helper /etc/sudoers.d/ Step 3: Configure Minikube Defaults # To prevent Minikube from accidentally trying to wake up your VMs with the default Docker driver in the future (which will throw a GUEST_DRIVER_MISMATCH error), explicitly tell your global config to always use vfkit:\nminikube config set driver vfkit Step 4: Start the Cluster (With Extra Memory!) # This is the most critical step for Apple Silicon users. A multi-node vfkit cluster requires at least 3072 MB of memory per node. If you use the default 2048 MB, the VMs will silently crash during boot, and Minikube will incorrectly blame your firewall or bootpd for failing to assign an IP address.\nStart the cluster:\nminikube start --network=vmnet-shared --nodes=2 --cpus=2 --memory=3072 Because we set the default driver in Step 3, we no longer need to pass the --driver=vfkit flag here!\nStep 5: Verify Networking and Enable Ingress # Check that macOS successfully handed out direct, host-routable IP addresses to your nodes:\nkubectl get nodes -o wide You should see IPs in the 192.168.64.x range instead of standard localhost IPs.\nEnable the NGINX Ingress controller:\nminikube addons enable ingress Step 6: Deploy a Test Application # To prove direct routing works, let\u0026rsquo;s deploy a modern, ARM-compatible web server and route a local domain to it.\n1. Deploy Alpine NGINX and expose it internally:\nkubectl create deployment hello-app --image=nginx:alpine kubectl expose deployment hello-app --port=80 2. Create the Ingress rule:\nThis tells the Ingress controller to route traffic for hello.local to our new app.\nkubectl apply -f - \u0026lt;\u0026lt;EOF apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: hello-ingress spec: ingressClassName: nginx rules: - host: hello.local http: paths: - path: / pathType: Prefix backend: service: name: hello-app port: number: 80 EOF Step 7: Update Your Mac\u0026rsquo;s Hosts File # You need to tell your Mac\u0026rsquo;s DNS to route hello.local to the IP address of your Minikube control-plane node.\n1. Find your Minikube IP:\nminikube ip Note this IP — usually 192.168.64.2.\n2. Edit your hosts file:\nsudo nano /etc/hosts Add this line to the bottom (replacing the IP with your actual Minikube IP):\n192.168.64.2 hello.local Save and exit.\nStep 8: Test the Connection # You can now access your cluster directly without any port-forwarding!\nOpen your web browser and navigate strictly to: http://hello.local\n🚨 Troubleshooting: The \u0026ldquo;0-Byte Plist\u0026rdquo; Bug # If Minikube gets stuck in a loop complaining that it could not find an IP address, and you previously tried to fix it by creating a blank /var/db/dhcpd_leases file, you likely broke the macOS DHCP server.\nmacOS requires that file to be a valid Apple XML Property List. If it is 0 bytes, bootpd crashes entirely. Run this to inject the correct empty XML structure and restart the DHCP server:\nsudo bash -c \u0026#39;cat \u0026gt; /var/db/dhcpd_leases \u0026lt;\u0026lt;EOF \u0026lt;?xml version=\u0026#34;1.0\u0026#34; encoding=\u0026#34;UTF-8\u0026#34;?\u0026gt; \u0026lt;!DOCTYPE plist PUBLIC \u0026#34;-//Apple//DTD PLIST 1.0//EN\u0026#34; \u0026#34;http://www.apple.com/DTDs/PropertyList-1.0.dtd\u0026#34;\u0026gt; \u0026lt;plist version=\u0026#34;1.0\u0026#34;\u0026gt; \u0026lt;dict\u0026gt; \u0026lt;/dict\u0026gt; \u0026lt;/plist\u0026gt; EOF\u0026#39; sudo killall bootpd ","date":"20 June 2026","externalUrl":null,"permalink":"/minikube-ingress-on-macos-with-vfkit/","section":"Blog","summary":"","title":"Minikube + Ingress on macOS with vfkit","type":"posts"},{"content":"","date":"20 June 2026","externalUrl":null,"permalink":"/tags/networking/","section":"Tags","summary":"","title":"Networking","type":"tags"},{"content":"","date":"20 June 2026","externalUrl":null,"permalink":"/tags/nginx/","section":"Tags","summary":"","title":"Nginx","type":"tags"},{"content":"","date":"20 June 2026","externalUrl":null,"permalink":"/tags/vfkit/","section":"Tags","summary":"","title":"Vfkit","type":"tags"},{"content":"","date":"15 December 2025","externalUrl":null,"permalink":"/tags/containers/","section":"Tags","summary":"","title":"Containers","type":"tags"},{"content":"","date":"15 December 2025","externalUrl":null,"permalink":"/tags/docker/","section":"Tags","summary":"","title":"Docker","type":"tags"},{"content":" The Kubernetes API # The core of Kubernetes\u0026rsquo; control plane is the API server. The API server exposes an HTTP API that lets end users, different parts of your cluster, and external components communicate with one another.\nThe Kubernetes API lets you query and manipulate the state of API objects in Kubernetes (for example: Pods, Namespaces, ConfigMaps, and Events).\nMost operations can be performed through the kubectl command-line interface or other command-line tools, such as kubeadm, which in turn use the API. However, you can also access the API directly using REST calls.\n","date":"15 December 2025","externalUrl":null,"permalink":"/kubernetes-101-advanced-kubernetes-using-the-api/","section":"Blog","summary":"","title":"Kubernetes 101: Advanced Kubernetes - Using the API","type":"posts"},{"content":"","date":"15 December 2025","externalUrl":null,"permalink":"/tags/kubernetes-api/","section":"Tags","summary":"","title":"Kubernetes API","type":"tags"},{"content":" Providing Variables to Kubernetes Applications # While we shouldn\u0026rsquo;t run naked Pods, we\u0026rsquo;ve already seen we can pass environment variables when creating a Pod:\nkubectl run mydb --image=mysql --env=\u0026quot;MYSQL_ROOT_PASSWORD=password\u0026quot;\nWhen creating a Deployment, however, there\u0026rsquo;s no command line option to provide variables. We\u0026rsquo;ll need to create the Deployment first, then set the environment variables:\nkubectl create deploy mydb --image=mysql kubectl set env deploy mydb MYSQL_ROOT_PASSWORD=password Obviously you could generate the Deployment YAML file first and add your variables to the YAML file before creating the Deployment.\nstudent@minikube:~$ kubectl create deployment mydb --image=mariadb deployment.apps/mydb created student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE mydb-fb7ff4d78-kqbvj 0/1 Error 0 40s student@minikube:~$ kubectl logs mydb-fb7ff4d78-kqbvj 2022-04-16 07:41:41+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.7.3+maria~focal started. 2022-04-16 07:41:41+00:00 [Note] [Entrypoint]: Switching to dedicated user \u0026#39;mysql\u0026#39; 2022-04-16 07:41:41+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.7.3+maria~focal started. 2022-04-16 07:41:41+00:00 [ERROR] [Entrypoint]: Database is uninitialized and password option is not specified You need to specify one of MARIADB_ROOT_PASSWORD, MARIADB_ALLOW_EMPTY_ROOT_PASSWORD and MARIADB_RANDOM_ROOT_PASSWORD student@minikube:~$ kubectl set env deploy mydb MYSQL_ROOT_PASSWORD=password deployment.apps/mydb env updated student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE mydb-6df85bcdbb-thm2h 0/1 ContainerCreating 0 5s mydb-fb7ff4d78-kqbvj 0/1 Error 3 (47s ago) 108s student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE mydb-6df85bcdbb-thm2h 1/1 Running 0 13s student@minikube:~$ kubectl get deploy mydb -o yaml \u0026gt; mydb.yml ... student@minikube:~$ kubectl create deploy mynewdb --image=mariadb --dry-run=client -o yaml \u0026gt; mynewdb.yaml student@minikube:~$ kubectl create -f mynewdb.yaml deployment.apps/mynewdb created student@minikube:~$ kubectl set env deploy mynewdb MYSQL_ROOT_PASSWORD=password --dry-run=client -o yaml \u0026gt; mynewdb.yaml student@minikube:~$ grep -i password mynewdb.yaml - name: MYSQL_ROOT_PASSWORD value: password student@minikube:~$ kubectl describe deploy mynewdb | grep -i password student@minikube:~$ kubectl apply -f mynewdb.yaml deployment.apps/mynewdb configured student@minikube:~$ kubectl describe deploy mynewdb | grep -i password MYSQL_ROOT_PASSWORD: password ConfigMaps # Code should be static, which makes it portable so that it can be used in other environments. To achieve this we need to separate site-specific information, like environment variables, from the code. These should not be provided in the Deployment configuration.\nConfigMaps are the solution to this issue, we can define variables and have our Deployment point to the ConfigMap. ConfigMaps are created in a different way depending what it will be used for:\nVariables Configuration Files Command line arguments Providing Variables with ConfigMaps # We can create a ConfigMap for variables in two ways:\nBy passing a file that contains the variables in a key=value format:\nkubectl create cm mycm --from-env-file=myfile By passing the variables directly:\nkubectl create cm mycm--from-literal=MYSQL_ROOT_PASSWORD=password Once you have the ConfigMap, you can update your deployment so that it points to the ConfigMap: kubectl set env --from=configmap/mycm deploy/mydeployment\nstudent@minikube:~$ cat dbvarsfile MYSQL_ROOT_PASSWORD=password MYSQL_USER=joeri student@minikube:~$ kubectl create cm mydbvars --from-env-file=dbvarsfile configmap/mydbvars created student@minikube:~$ kubectl create deploy mydb --image=mariadb deployment.apps/mydb created student@minikube:~$ kubectl set env deploy mydb --from=configmap/mydbvars deployment.apps/mydb env updated student@minikube:~$ kubectl describe deploy mydb | grep MYSQL_ MYSQL_ROOT_PASSWORD: \u0026lt;set to the key \u0026#39;MYSQL_ROOT_PASSWORD\u0026#39; of config map \u0026#39;mydbvars\u0026#39;\u0026gt; Optional: false MYSQL_USER: \u0026lt;set to the key \u0026#39;MYSQL_USER\u0026#39; of config map \u0026#39;mydbvars\u0026#39;\u0026gt; Optional: false student@minikube:~$ kubectl get deploy mydb -o yaml \u0026gt; mydb.yaml ... Providing Configuration Files with ConfigMaps # In addition to providing variables, we can provide configuration files to our application by making use of ConfigMaps:\nkubectl create cm myconf --from-file=/my/file.conf\nIf a ConfigMap is created from a directory instead of a file, all files in that directory will be included in the ConfigMap. When using ConfigMap for configuration files the ConfigMap must be mounted in the application, it behaves similarly to a Volume.\nFrom a high level, we need to:\nGenerate the base YAML code, then add the ConfigMap mount to it later Define a Volume of the ConfigMap type in the application manifest Mount this volume on a specific directory, the configuration file will appear inside that directory. In the below example we\u0026rsquo;ll provide an index.html file to Nginx via a ConfigMap:\nstudent@minikube:~$ echo \u0026#34;Hello World!\u0026#34; \u0026gt; index.html student@minikube:~$ kubectl create cm myindex --from-file=index.html configmap/myindex created student@minikube:~$ kubectl describe cm myindex Name: myindex Namespace: default Labels: \u0026lt;none\u0026gt; Annotations: \u0026lt;none\u0026gt; Data ==== index.html: ---- Hello World! BinaryData ==== Events: \u0026lt;none\u0026gt; student@minikube:~$ kubectl create deploy myweb --image=nginx deployment.apps/myweb created We\u0026rsquo;ll edit the deployment and add volumes and volumeMounts to spec.template.spec:\nstudent@minikube:~$ kubectl edit deploy myweb ... spec: volumes: - name: cmvol configMap: name: myindex containers: volumeMounts: - mountPath: /usr/share/nginx/html name: cmvol - image: nginx imagePullPolicy: Always name: nginx resources: {} ... Let\u0026rsquo;s verify our changes:\nstudent@minikube:~$ kubectl describe deploy myweb Pod Template: Labels: app=myweb Containers: nginx: Image: nginx Port: \u0026lt;none\u0026gt; Host Port: \u0026lt;none\u0026gt; Environment: \u0026lt;none\u0026gt; Mounts: /usr/share/nginx/html from cmvol (rw) Volumes: cmvol: Type: ConfigMap (a volume populated by a ConfigMap) Name: myindex ... student@minikube:~$ kubectl get all --selector app=myweb NAME READY STATUS RESTARTS AGE pod/myweb-ff8bf9988-287n2 1/1 Running 0 13m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/myweb 1/1 1 1 19m NAME DESIRED CURRENT READY AGE replicaset.apps/myweb-8764bf4c8 0 0 0 19m replicaset.apps/myweb-ff8bf9988 1 1 1 13m student@minikube:~$ kubectl exec pod/myweb-ff8bf9988-287n2 -- cat /usr/share/nginx/html/index.html Hello World! Understanding Secrets # Secrets allow you to store sensitive data such as passwords, authentication tokens and SSH keys, outside of a Pod to reduce the risk of accidental expose. Some Secrets are automatically created by Kubernetes while others can be created by the user. System-created Secrets are important for Kubernetes resources to connect to other cluster resources.\nSecrets are Base64 encoded and not encrypted.\nThree types of Secret types are offered:\ndocker-registry: Used for connecting to a Docker registry. TLS: Used to store TLS key material. generic: Creates a secret from a local file, directory or literal value You need to specify the type when defining the Secret: kubectl create secret generic ...\nHow Kubernetes Uses Secrets # All Kubernetes resources need access to TLS keys in order to access the Kubernetes API. These keys are provided by Secrets and used through ServiceAccounts. By using the ServiceAccount, the application has access to its Secret.\nLet\u0026rsquo;s inspect one of the secrets Kubernetes uses. As mentioned previously, Secrets are used through ServiceAccounts, so we need to find out the ServiceAccount first before we can inspect the details of the Secret:\nstudent@minikube:~$ kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-64897985d-lhqq6 1/1 Running 1 (6m49s ago) 25m etcd-minikube 1/1 Running 1 (6m49s ago) 25m kube-apiserver-minikube 1/1 Running 1 (6m49s ago) 25m kube-controller-manager-minikube 1/1 Running 1 (6m49s ago) 25m kube-proxy-khgjl 1/1 Running 1 (6m49s ago) 25m kube-scheduler-minikube 1/1 Running 1 (6m49s ago) 25m storage-provisioner 1/1 Running 2 (6m49s ago) 25m student@minikube:~$ kubectl get pods -n kube-system coredns-64897985d-lhqq6 -o yaml | grep serviceAccount serviceAccount: coredns serviceAccountName: coredns - serviceAccountToken: student@minikube:~$ kubectl get sa -n kube-system coredns -o yaml apiVersion: v1 kind: ServiceAccount metadata: creationTimestamp: \u0026#34;2022-02-01T15:48:54Z\u0026#34; name: coredns namespace: kube-system resourceVersion: \u0026#34;299\u0026#34; uid: 519a806e-35c0-45be-a5a0-495d9f7c7586 secrets: - name: coredns-token-j6qdj student@minikube:~$ kubectl get secret -n kube-system coredns-token-j6qdj -o yaml apiVersion: v1 data: ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCakNDQWU2Z0F3SUJBZ0lCQVRBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwdGFXNXAKYTNWaVpVTkJNQjRYRFRJeU1ETXdOekUxTlRrd05Wb1hEVE15TURNd05URTFOVGt3TlZvd0ZURVRNQkVHQTFVRQpBeE1LYldsdWFXdDFZbVZEUVRDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS0pDCnhPam5XYXRKSW9tdUcrSGtsM3J0aFhGV0NwUGhic3FXTkhsbGlqeTlWWVRvYlYwdHVrUW1sRkRvZGc2N1RULzQKWmlYNVFvUXdyV0NOSTYrYmtPMGpGMUhPRXJQNUF2S3ZJMEpabzliSTZzN1NPVmVsNHJsRGtRUGFScjBWajhrZwpGZTZNb2tUZGswQlBmQ1l5c2hhNmNBUGNaaHl1Wjl3clJRYi83dnZkS3BzZ2tLZ1ZOMmVEQnNqRzNGWFc1M2JvCkx6azJsT1NORHRxNndVSTdlZzIrNjR2UEQ5YkdWU09IU3JraVNMTVdtU3ZWL0d3SlV3dFd6YVhtZWhJZ1NLRVAKY3ZxMWtRN0dvUEVzTUF6TUtMb2F4bXdpZlUxQ0xISE93akhWTlZvVXcvVmNOQlZCOGlnRGd4cmJMSjg3bU9pOQpqbzJpck1BNTZqZExPVk1rUFlzQ0F3RUFBYU5oTUY4d0RnWURWUjBQQVFIL0JBUURBZ0trTUIwR0ExVWRKUVFXCk1CUUdDQ3NHQVFVRkJ3TUNCZ2dyQmdFRkJRY0RBVEFQQmdOVkhSTUJBZjhFQlRBREFRSC9NQjBHQTFVZERnUVcKQkJTODl5UHdEYzJxZG13VGFlbWxZcndvclRqVTdqQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFlOTdNNjV3WQpxUU5nR2NzT3A4Tm4rbzdGdXQ0cWMyWldjdll5bEZKUnFURjFIVjhwZDIzTFR0V3VoRkQraVk5SDJuLzFNdzdwCnVFcHdVUjAzVHpIUUVpL1JjTUJPV0JBakFGVzJHck5RelhVbzdyOE03a3FHdEN3MVd4WXduQVBhNGJ1SG41SWcKT0lhQTA4V25udW4rcFFRMW5WL25aU04yV2xwRzRrblhGcHAzcjhTQ21uVkd1L296VjV3bGZ3WU9Ea3prZExSMgp4bjA5SHhTWkJsclpDdFZqWUxDaVRYbkN3Q3pTVXZSNjhYWkNZVWRWTHF5ZzZyZXBYb0dsSkJzY0ZMZURtKzZrCnVZR1ZvY0Ezd0FpWktoazJPV2EzcGdnTFJod2xTdTRqaFZZNk82WFpvQXMzOHcvYzFTeWY1WDBqcFB1OVdPRW8KSDdsTEpCOTdhamhYdVE9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== namespace: a3ViZS1zeXN0ZW0= token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklsWlplWEJ4UkdoUVNUQnJaM1JTTjJGNVdUQTRlakJZUjBWS2VVaHdNRlJSYzFoQ1JFOXJPWGRGYmxFaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUpyZFdKbExYTjVjM1JsYlNJc0ltdDFZbVZ5Ym1WMFpYTXVhVzh2YzJWeWRtbGpaV0ZqWTI5MWJuUXZjMlZqY21WMExtNWhiV1VpT2lKamIzSmxaRzV6TFhSdmEyVnVMV28yY1dScUlpd2lhM1ZpWlhKdVpYUmxjeTVwYnk5elpYSjJhV05sWVdOamIzVnVkQzl6WlhKMmFXTmxMV0ZqWTI5MWJuUXVibUZ0WlNJNkltTnZjbVZrYm5NaUxDSnJkV0psY201bGRHVnpMbWx2TDNObGNuWnBZMlZoWTJOdmRXNTBMM05sY25acFkyVXRZV05qYjNWdWRDNTFhV1FpT2lJMU1UbGhPREEyWlMwek5XTXdMVFExWW1VdFlUVmhNQzAwT1RWa09XWTNZemMxT0RZaUxDSnpkV0lpT2lKemVYTjBaVzA2YzJWeWRtbGpaV0ZqWTI5MWJuUTZhM1ZpWlMxemVYTjBaVzA2WTI5eVpXUnVjeUo5LmN6cGp6SUM5NG9jSV81N21vZU5wZ2xXLWtVZnpHdUlUUktfa09qbkw3M0xuN1p2M2tLMWU2TjNqbUpPSW95d2RMcms5NWNwZC1pT1VjQWdpQVcxN3dJRUZ1THR4WnVkbmsyNnBwWU1sdDNLWHpBMkJycjdkYzZGM0xjdG9RNTdPMEY0MnEybXpJS0dnVDBVYkhmYTNwTjd4ZDY0Zk04RVFpZUc2bEZBSlNuYjlBTGVqSjd6X1JjeWdkLU1SOE9Qc2gtd05KMW1RSlVrUktzenVwTHdZcERKSXVCSGx6a093Rm04YXJ5ODZ3Y0pGdzNSbm5mcFo4ZTF0aWwtWUVSTmV3aDdMdzhvTGRrSzJNUnVVSnBKZmtGZ1kteWhWejdwa3MtNW53U05BUWVpTEk2RG9oUFBqd3BYa3hzWWVCbUhZVWo1b3JMNlZ5NG9Xb1ZZTnJIU0JvUQ== kind: Secret metadata: annotations: kubernetes.io/service-account.name: coredns kubernetes.io/service-account.uid: 519a806e-35c0-45be-a5a0-495d9f7c7586 creationTimestamp: \u0026#34;2022-02-01T15:48:55Z\u0026#34; name: coredns-token-j6qdj namespace: kube-system resourceVersion: \u0026#34;294\u0026#34; uid: d5b84f10-e6fa-46d0-92ec-2b7895a799eb type: kubernetes.io/service-account-token Notice how the values in the Secret Yaml output above are base64 encoded, e.g. for the namespace:\nstudent@minikube:~$ echo a3ViZS1zeXN0ZW0= | base64 -d kube-system Configuring Applications to Use Secrets # There are different use cases for using Secrets in applications:\nProviding TLS keys to the application:\nkubectl create secret tls my-tls-keys --cert=pathto/my.crt --key=pathto/my.key Provide security to passwords:\nkubectl create generic my-secret-pw --from-literal=password=verysecret Provide access to an SSH private key:\nkubectl create generic my-ssh-key --from-file=ssh-private-key=.ssh/id_rsa Provide access to sensitive files which would be mounted in the application with root access only:\nkubectl create secret generic my-secret-file --from-file=/my/secretfile Secrets are used in a similar way to using ConfigMaps in applications:\nIf your Secret contains variables (like a password), use kubectl set env. If it contains files (like keys), mount the Secret. Consider using defaultMode: 0400 permissions when mounting the Secret in the Pod spec. Mounted Secrets are automatically updated in the application when the Secret is updated.\nLet\u0026rsquo;s demonstrate this:\nstudent@minikube:~$ kubectl create secret generic dbpw --from-literal=ROOT_PASSWORD=password secret/dbpw created student@minikube:~$ kubectl describe secret dbpw Name: dbpw Namespace: default Labels: \u0026lt;none\u0026gt; Annotations: \u0026lt;none\u0026gt; Type: Opaque Data ==== ROOT_PASSWORD: 8 bytes student@minikube:~$ kubectl get secret dbpw -o yaml apiVersion: v1 data: ROOT_PASSWORD: cGFzc3dvcmQ= kind: Secret metadata: creationTimestamp: \u0026#34;2022-02-01T16:30:07Z\u0026#34; name: dbpw namespace: default resourceVersion: \u0026#34;1661\u0026#34; uid: 6aff6adf-73e1-4ffd-b99a-fd036a034c6b type: Opaque student@minikube:~$ echo cGFzc3dvcmQ= | base64 -d password Now, let\u0026rsquo;s deploy our Secret to an app:\nstudent@minikube:~$ kubectl create deployment mynewdb --image=mariadb deployment.apps/mynewdb created Remember that mariadb is expecting at the very least a MYSQL_ROOT_PASSWORD environment variable. But since we created our Secret with ROOT_PASSWORD instead of MYSQL_ROOT_PASSWORD we would need to set a prefix when attaching the Secret to the application. This can come in handy in case I have other applications that could potentially use the same Secret.\nstudent@minikube:~$ kubectl set env deployment mynewdb --from=secret/dbpw --prefix=MYSQL_ deployment.apps/mynewdb env updated Now, while our password is base64 encoded, this isn\u0026rsquo;t the case inside the Pod where it\u0026rsquo;s in clear text:\nstudent@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE mynewdb-7cc5fb9c55-58wkz 1/1 Running 0 13m student@minikube:~$ kubectl exec mynewdb-7cc5fb9c55-58wkz -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=mynewdb-7cc5fb9c55-58wkz MYSQL_ROOT_PASSWORD=password Configuring Docker Registry Access Secret # The docker-registry Secret type stores container registry (Docker Hub, Quay.io, self hosted, \u0026hellip;) authentication credentials. While you don\u0026rsquo;t need to authenticate, it\u0026rsquo;s recommended to prevent pull rate errors in case you\u0026rsquo;re running a busy cluster.\nThere\u0026rsquo;s two ways to create the Secret: Either by directly passing the credentials, or by passing an existing Docker Config file which contains the credentials:\nstudent@minikube:~$ kubectl create secret docker-registry -h Examples: # If you don\u0026#39;t already have a .dockercfg file, you can create a dockercfg secret directly by using: kubectl create secret docker-registry my-secret --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL # Create a new secret named my-secret from ~/.docker/config.json kubectl create secret docker-registry my-secret --from-file=.dockerconfigjson=path/to/.docker/config.json ","date":"28 August 2025","externalUrl":null,"permalink":"/kubernetes-101-building-scalable-applications-configmaps-secrets/","section":"Blog","summary":"","title":"Kubernetes 101: Building Scalable Applications - ConfigMaps \u0026 Secrets","type":"posts"},{"content":" Storage Options # Files stored in a container will only live as long as the container itself: they are ephemeral. To solve this problem we can use Pod Volumes, they outlive containers and stay available during the Pod lifetime. The Pod Volume is a property of the Pod, not the container.\nPod Volumes can directly bind to any specific storage type, e.g. Cephs, emptyDir, fibre channel, NFS, \u0026hellip; By using Persistent Volume Claims, you can decouple the Pod from site-specific storage: You make the Pod specification more portable since you don\u0026rsquo;t configure the site-specific storage but only describe what\u0026rsquo;s needed from the storage: Size and permissions.\nThe Persistent Volume Claim connects to a Persistent Volume which in turn defines access to external storage available in the cluster. A site administrator must make sure this Persistent Volume exists. So when a Persistent Volume Claim is created, it will search for an available Persistent Volume that matches the requirements of the storage request in the Persistent Volume Claim. If no match is found, there\u0026rsquo;s StorageClass that can automatically create and allocate the storage.\nThis abstraction allows a developer to create and distribute generic Pod manifest files and leave the storage up to the site where it\u0026rsquo;s being deployed. We\u0026rsquo;ll go over examples of this to make the concept more clear.\nConfiguring Volume Storage # Pod local volumes are defined in pod.spec.volumes, they point to a specific volume type but for testing purposes emptyDir and hostPath are common. This volume is mounted through pod.spec.containers.volumeMounts.\napiVersion: v1 kind: Pod metadata: name: volpod spec: volumes: - name: test emptyDir: {} containers: - name: centos1 image: centos:7 command: - sleep - \u0026#34;3600\u0026#34; volumeMounts: - mountPath: /centos1 name: test - name: centos2 image: centos:7 command: - sleep - \u0026#34;3600\u0026#34; volumeMounts: - mountPath: /centos2 name: test In the above Pod Spec, we\u0026rsquo;ve defined a volume named test with the volume type emptyDir. This volume is mounted in two containers on the /centos1 and /centos2 path inside the container. Both containers can share data via this volume:\nstudent@minikube:~$ kubectl create -f volpod.yaml pod/volpod created ... student@minikube:~$ kubectl exec -it volpod -c centos1 -- bash -c \u0026#39;echo \u0026#34;Hi there!\u0026#34; \u0026gt; /centos1/hello\u0026#39; student@minikube:~$ kubectl exec -it volpod -c centos2 -- cat /centos2/hello Hi there! Persistent Volume Storage # A Persistent Volume is a resource that exists independently from any Pod, it ensures that data is kept during container or Pod restarts. We\u0026rsquo;ll use a Persistent Volume Claim to connect to a Persistent Volume. The Persistent Volume Claim is what actually talks to the backend storage provider and it will use volumes available on that storage type: It will search for available volumes depending on the requested capacity and access mode.\nkind: PersistentVolume apiVersion: v1 metadata: name: pv-volume labels: type: local spec: capacity: storage: 2Gi accessModes: - ReadWriteOnce hostPath: path: \u0026#34;/mydata\u0026#34; In the above PersistentVolume we\u0026rsquo;ve created a Persistent Volume resource with the name pv-volume, a capacity of 2GB, an accessMode of ReadWriteOnce and a hostPath of /mydata. The hostPath is created on the worker-node where the Pod that will use this PersistentVolume is running. ReadWriteOnce makes sure that only one Pod can read/write data at the same time, ReadWriteMany or ReadOnly can be used as well.\nstudent@minikube:~$ kubectl create -f pv.yaml persistentvolume/pv-volume created student@minikube:~$ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-volume 2Gi RWO Retain Available 70s student@minikube:~$ kubectl describe pv pv-volume Name: pv-volume Labels: type=local Annotations: \u0026lt;none\u0026gt; Finalizers: [kubernetes.io/pv-protection] StorageClass: Status: Available Claim: Reclaim Policy: Retain Access Modes: RWO VolumeMode: Filesystem Capacity: 2Gi Node Affinity: \u0026lt;none\u0026gt; Message: Source: Type: HostPath (bare host directory volume) Path: /mydata HostPathType: Events: \u0026lt;none\u0026gt; Configuring Persistent Volume Claims # To use a Persistent Volume, we need a Persistent Volume Claim which requests access to Persistent Volume. The Pod Volume spec uses the name of the Persistent Volume Claim and in turn the PVC accesses the Persistent Volume. After connecting to a Persistent Volume, the Persistent Volume Claim will show as bound. The bind is exclusive, the Persistent Volume cannot be used by another Persistent Volume Claim.\nkind: PersistentVolumeClaim apiVersion: v1 metadata: name: pv-claim spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi Notice that in the above spec, the PVC does not connect to a specific Persistent Volume. The only thing we see is that we need a volume with a capacity of 1GB and ReadWriteOnce access.\nstudent@minikube:~$ kubectl create -f pvc.yaml persistentvolumeclaim/pv-claim created student@minikube:~$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE pv-claim Bound pvc-d5f02edb-3d71-4a69-b977-71fd5bfa020e 1Gi RWO standard 22s student@minikube:~$ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-volume 2Gi RWO Retain Available 11m pvc-d5f02edb-3d71-4a69-b977-71fd5bfa020e 1Gi RWO Delete Bound default/pv-claim standard 39s Our pv-volume Persistent Volume was not used and instead a new Persistent Volume was created by StorageClass because there was no available match due to the capacity request in the PVC.\nPod Storage with PV and PVC # The purpose of configuring a Pod with a Persistent Volume Claim is to decouple from site-specific information: When distributing a Pod spec with a PVC spec we do not need to know anything about site-specific storage. The PVC will find the necessary Persistent Volume storage to bind to:\n--- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: nginx-pvc spec: accessModes: - ReadWriteMany resources: requests: storage: 2Gi --- kind: Pod apiVersion: v1 metadata: name: nginx-pvc-pod spec: volumes: - name: site-storage persistentVolumeClaim: claimName: nginx-pvc containers: - name: pv-container image: nginx ports: - containerPort: 80 name: webserver volumeMounts: - mountPath: \u0026#34;/usr/share/nginx/html\u0026#34; name: site-storage student@minikube:~$ kubectl create -f ckad/pvc-pod.yaml persistentvolumeclaim/nginx-pvc created pod/nginx-pvc-pod created student@minikube:~$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE nginx-pvc Bound pvc-b7327501-ff4c-4f6d-9c79-d10c6ce771e8 2Gi RWX standard 13s student@minikube:~$ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-b7327501-ff4c-4f6d-9c79-d10c6ce771e8 2Gi RWX Delete Bound default/nginx-pvc standard 9s student@minikube:~$ kubectl describe pv pvc-b7 Name: pvc-b7327501-ff4c-4f6d-9c79-d10c6ce771e8 Labels: \u0026lt;none\u0026gt; Annotations: hostPathProvisionerIdentity: 3d1fa9ec-a297-4851-8782-97e7eb238447 pv.kubernetes.io/provisioned-by: k8s.io/minikube-hostpath Finalizers: [kubernetes.io/pv-protection] StorageClass: standard Status: Bound Claim: default/nginx-pvc Reclaim Policy: Delete Access Modes: RWX VolumeMode: Filesystem Capacity: 2Gi Node Affinity: \u0026lt;none\u0026gt; Message: Source: Type: HostPath (bare host directory volume) Path: /tmp/hostpath-provisioner/default/nginx-pvc HostPathType: Events: \u0026lt;none\u0026gt; student@minikube:~$ kubectl exec -it nginx-pvc-pod -- touch /usr/share/nginx/html/testfile student@minikube:~$ minikube ssh docker@minikube:~$ ls /tmp/hostpath-provisioner/default/nginx-pvc/ testfile StorageClass # Kubernetes StorageClass allows for automatic provisioning of Persistent Volumes when a Persistent Volume Claim request comes in. This must be backed by a Storage Provisioner which ultimately takes care of the volume configuration.\nStorageClass can also be used a a selector label with the storageClassName field. Normally, PVC to PV binding is done on best match.\n--- apiVersion: v1 kind: PersistentVolume metadata: name: task-pv-volume labels: type: local spec: storageClassName: manual capacity: storage: 2Gi accessModes: - ReadWriteMany hostPath: path: \u0026#34;/mnt/data\u0026#34; --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: task-pv-claim spec: storageClassName: manual accessModes: - ReadWriteMany resources: requests: storage: 2Gi --- apiVersion: v1 kind: Pod metadata: name: task-pv-pod spec: volumes: - name: task-pv-storage persistentVolumeClaim: claimName: pv-claim containers: - name: task-pv-container image: httpd ports: - containerPort: 80 name: \u0026#34;httpd-server\u0026#34; volumeMounts: - mountPath: \u0026#34;/var/www/html\u0026#34; name: task-pv-storage ","date":"1 June 2025","externalUrl":null,"permalink":"/kubernetes-101-building-scalable-applications-storage/","section":"Blog","summary":"","title":"Kubernetes 101: Building Scalable Applications - Storage","type":"posts"},{"content":"","date":"1 June 2025","externalUrl":null,"permalink":"/tags/persistent-volume/","section":"Tags","summary":"","title":"Persistent Volume","type":"tags"},{"content":"","date":"1 June 2025","externalUrl":null,"permalink":"/tags/persistent-volume-claim/","section":"Tags","summary":"","title":"Persistent Volume Claim","type":"tags"},{"content":"","date":"1 June 2025","externalUrl":null,"permalink":"/tags/pod/","section":"Tags","summary":"","title":"Pod","type":"tags"},{"content":"","date":"1 June 2025","externalUrl":null,"permalink":"/tags/storage/","section":"Tags","summary":"","title":"Storage","type":"tags"},{"content":"","date":"1 June 2025","externalUrl":null,"permalink":"/tags/storageclass/","section":"Tags","summary":"","title":"StorageClass","type":"tags"},{"content":" The Kubernetes network model dictates that:\nEvery Pod has its own IP address Containers within a Pod share the Pod IP address and can communicate with each other using a loopback interface (localhost). Pods can communicate with all other Pods in the cluster using the Pod IP addresses and without using NAT. Isolation is defined by using network policies. Pod-to-Pod communication is the foundation of Kubernetes. You can look at a Pod like you would look at a VM, the VM has a unique IP address. The containers within the Pods are like processes running within a VM, they run in the same network namespace and share an IP address.\nBasic network connectivity is built-in with kubenet but can be extended by using third-party network implementations that plug into Kubernetes using the Container Network Interface API.\nThe Kubernetes networking model relies heavily on IP addresses. Services, Pods, containers, and nodes communicate using IP addresses and ports:\nClusterIP: The IP address assigned to a Service. This address is stable for the lifetime of the Service. Pod IP: The IP address assigned to a given Pod. This is ephemeral. Node IP: The IP address assigned to a given node. Services # A Service is an API resource that is used to expose a logical set of Pods, determined by a selector (label), to an external network by applying round-robin load balancing that forwards the traffic. kube-controller-manager will continuously scan for Pods that match a selector and include those in the Service. Adding or removing Pods immediately impacts the Service.\nServices exist independently from the applications or Pods they provide access to, e.g. removing a Deployment will not remove a Service. This means that one Service can provide access to Pods in multiple Deployments, Kubernetes will automatically load balance between these Pods.\nkube-proxy on the nodes watches the Kubernetes API for new Services and endpoints (connected Pods). It opens random ports and listens for traffic to the Service port on the Cluster IP address, then redirects traffic to a Pod that is specified as an endpoint. It typically doesn\u0026rsquo;t require any configuration.\nThere are different Service Types:\nClusterIP: The default type which exposes the Service on an internal cluster IP address. NodePort: Opens a specific port on the node that forwards to the Service cluster IP address. LoadBalancer: Used on public cloud, it will provision a load balancer in the cloud for the Service. ExternalName: Works with DNS names. We will focus on ClusterIP and NodePort.\nCreating Services # kubectl expose can be used to create Services, providing access to Deployments, ReplicaSets, Pods or other. In most cases it exposes a Deployment which in turn allocates its Pods as the Service Endpoint. If you inspect the Service, you\u0026rsquo;ll see it doesn\u0026rsquo;t actually connect to the Deployment but to the Pods in the Deployment by using the Selector label. The --port argument is required to specify the port that the Service should use.\nThere are different types of ports in Services:\nport: The port on which the Service is accessible. targetport: The port on the application that the Service addresses. The same value for port will be used if targetport is not specified. nodeport: The port that is exposed externally while using the nodePort Service type. Required when using the nodePort Service Type but is set automatically. kubectl create service can be used as an alternative solution to create Services. When creating a NodePort Service type, the port and targetport are specified as a key:value pair in the --tcp argument:\nkubectl create service nodeport my-node-port-service --tcp=80:80 Here we are not targeting a Deployment, but because I\u0026rsquo;m naming the NodePort Service my-node-port-service the service will look for all Pods that have the label selector app=my-node-port-service.\nLet\u0026rsquo;s expose a simple Nginx application.\nstudent@minikube:~$ kubectl create deploy nginx-app --image=nginx:latest --replicas=3 deployment.apps/nginx-app created student@minikube:~$ kubectl expose deploy nginx-app --port=80 service/nginx-app exposed student@minikube:~$ kubectl get service nginx-app NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-app ClusterIP 10.97.84.163 \u0026lt;none\u0026gt; 80/TCP 8s We\u0026rsquo;ve created a service of the type ClusterIP which is available on the internal IP address 10.97.84.163. The IP address is internal from the point of view of the Kubernetes cluster. Remember, we\u0026rsquo;re not working inside the cluster:\nstudent@minikube:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 4680f20c93ff gcr.io/k8s-minikube/kicbase:v0.0.30 \u0026#34;/usr/local/bin/entr…\u0026#34; 2 hours ago Up 2 hours 127.0.0.1:49157-\u0026gt;22/tcp, 127.0.0.1:49156-\u0026gt;2376/tcp, 127.0.0.1:49155-\u0026gt;5000/tcp, 127.0.0.1:49154-\u0026gt;8443/tcp, 127.0.0.1:49153-\u0026gt;32443/tcp minikube On our minikube machine, we have a minikube Docker container running which runs the Kubernetes cluster and node inside. This means that we cannot reach the ClusterIP address from outside of Docker. In order to achieve that, we need to open a port on our Kubernetes Node using the NodePort service type. Edit the service, change the type and add the nodePort value:\nstudent@minikube:~$ kubectl edit service nginx-app # Please edit the object below. Lines beginning with a \u0026#39;#\u0026#39; will be ignored, # and an empty file will abort the edit. If an error occurs while saving this file will be # reopened with the relevant failures. # apiVersion: v1 kind: Service metadata: creationTimestamp: null labels: app: nginx-app name: nginx-app namespace: default resourceVersion: \u0026#34;6160\u0026#34; uid: 8d2e2744-328d-4e1f-b8f8-96404515faae spec: clusterIP: 10.97.84.163 clusterIPs: - 10.97.84.163 externalTrafficPolicy: Cluster internalTrafficPolicy: Cluster ipFamilies: - IPv4 ipFamilyPolicy: SingleStack ports: - nodePort: 32000 port: 80 protocol: TCP targetPort: 80 selector: app: nginx-app sessionAffinity: None type: NodePort status: loadBalancer: {} Save your changes.\nstudent@minikube:~$ kubectl get service nginx-app NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-app NodePort 10.97.84.163 \u0026lt;none\u0026gt; 80:32000/TCP 4m8s We see that our Service Type has changed and the Service is running on port 80 accessible trough NodePort 32000:\nstudent@minikube:~$ curl http://$(minikube ip):32000 \u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;Welcome to nginx!\u0026lt;/title\u0026gt; The minikube ip command shows what IP address your Kubernetes node is using, in the above example I applied command substitution.\nkubectl create service can be used as an alternative solution to create Services. When creating a NodePort Service type, the port and targetport are specified as a key:value pair in the --tcp argument:\nkubectl create service nodeport nginx-app --tcp=80:80 As opposed to the kubectl expose deployment, here we are not targeting a Deployment, but because I\u0026rsquo;m naming the NodePort Service nginx-app the service will look for all Pods that have the label selector app=nginx-app which would be all the Pods in our nginx-app Deployment.\nstudent@minikube:~$ kubectl create service nodeport nginx-app --tcp=80:80 service/nginx-app created student@minikube:~$ kubectl describe service nginx-app Name: nginx-app Namespace: default Labels: app=nginx-app Annotations: \u0026lt;none\u0026gt; Selector: app=nginx-app Using Service Resources in Microservices # In a microservices architecture, different frontend and backend Pods are used to provide the application:\nFrontend Pods (e.g. webservers) can be exposed for external access using the NodePort Service type. Backend Pods (e.g. databases) can be exposed internally only using the clusterIP Service type. An example would be a frontend Deployment with WordPress and a backend Deployment with MariaDB. You don\u0026rsquo;t want to expose MariaDB to external traffic, only the frontend Pods should be able to communicate with the database. They can do so using the Cluster IP address, or even without IP address by using a headless ClusterIP Service type. We\u0026rsquo;ll cover that later on.\nServices and DNS # Exposed Services automatically register with the Kubernetes internal DNS. The internal DNS consists of the kube-dns Service and the coreDNS Pod.\nThis allows all Pods to address Services using the Service name:\nstudent@minikube:~$ kubectl get service,pods -n kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kube-dns ClusterIP 10.96.0.10 \u0026lt;none\u0026gt; 53/UDP,53/TCP,9153/TCP 3h39m NAME READY STATUS RESTARTS AGE pod/coredns-64897985d-2fwlb 1/1 Running 0 3h39m Notice the Cluster IP address of the kube-dns service above. Now, let\u0026rsquo;s run a Pod and have a look at its DNS settings:\nstudent@minikube:~$ kubectl run testpod --image=busybox -- sleep 3600 pod/testpod created student@minikube:~$ kubectl exec -it testpod -- cat /etc/resolv.conf nameserver 10.96.0.10 search default.svc.cluster.local svc.cluster.local cluster.local options ndots:5 The nameserver is set to the Cluster IP address of the kube-dns service. Lookups are also done in the default.svc.cluster.local domain, where default is the name of the Name Space:\nstudent@minikube:~$ kubectl exec -it testpod -- nslookup nginx-app Server:\t10.96.0.10 Address:\t10.96.0.10:53 Name:\tnginx-app.default.svc.cluster.local Address: 10.96.165.179 student@minikube:~$ kubectl get service nginx-app NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-app NodePort 10.96.165.179 \u0026lt;none\u0026gt; 80:32000/TCP 8m35s Ingress # Ingress is a Kubernetes API resource used to provide external access using DNS to internal Kubernetes cluster Services by means of an externally Ingress managed load balancer, also known as an Ingress Controller. Creating an Ingress resource without Ingress Controller has no effect, you need both. The Ingress Controller can be anything you\u0026rsquo;re already familiar with: HAProxy, Nginx, Apache, traefik, kong, \u0026hellip;\nTo summarize, Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource. Ingress can be configured to do the following:\nGive Services externally-reachable URLs Terminate SSL/TLS Load balance traffic Offer name based virtual hosting Configuring the Minikube Ingress Controller # Minikube provides an easy Ingress integration using a Minikube addon:\nstudent@minikube:~$ minikube addons list |-----------------------------|----------|--------------|--------------------------------| | ADDON NAME | PROFILE | STATUS | MAINTAINER | |-----------------------------|----------|--------------|--------------------------------| | ambassador | minikube | disabled | third-party (ambassador) | | auto-pause | minikube | disabled | google | | csi-hostpath-driver | minikube | disabled | kubernetes | | dashboard | minikube | disabled | kubernetes | | default-storageclass | minikube | enabled ✅ | kubernetes | | efk | minikube | disabled | third-party (elastic) | | freshpod | minikube | disabled | google | | gcp-auth | minikube | disabled | google | | gvisor | minikube | disabled | google | | helm-tiller | minikube | disabled | third-party (helm) | | ingress | minikube | disabled | unknown (third-party) | | ingress-dns | minikube | disabled | google | | istio | minikube | disabled | third-party (istio) | | istio-provisioner | minikube | disabled | third-party (istio) | | kong | minikube | disabled | third-party (Kong HQ) | | kubevirt | minikube | disabled | third-party (kubevirt) | | logviewer | minikube | disabled | unknown (third-party) | | metallb | minikube | disabled | third-party (metallb) | | metrics-server | minikube | disabled | kubernetes | | nvidia-driver-installer | minikube | disabled | google | | nvidia-gpu-device-plugin | minikube | disabled | third-party (nvidia) | | olm | minikube | disabled | third-party (operator | | | | | framework) | | pod-security-policy | minikube | disabled | unknown (third-party) | | portainer | minikube | disabled | portainer.io | | registry | minikube | disabled | google | | registry-aliases | minikube | disabled | unknown (third-party) | | registry-creds | minikube | disabled | third-party (upmc enterprises) | | storage-provisioner | minikube | enabled ✅ | google | | storage-provisioner-gluster | minikube | disabled | unknown (third-party) | | volumesnapshots | minikube | disabled | kubernetes | |-----------------------------|----------|--------------|--------------------------------| student@minikube:~$ minikube addons enable ingress 🌟 The \u0026#39;ingress\u0026#39; addon is enabled student@minikube:~$ kubectl get ns NAME STATUS AGE default Active 73m ingress-nginx Active 88s kube-node-lease Active 73m kube-public Active 73m kube-system Active 73m student@minikube:~$ kubectl get all -n ingress-nginx NAME READY STATUS RESTARTS AGE pod/ingress-nginx-admission-create-qz4sr 0/1 Completed 0 115s pod/ingress-nginx-admission-patch-tbzsw 0/1 Completed 1 115s pod/ingress-nginx-controller-cc8496874-nrsq6 1/1 Running 0 115s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/ingress-nginx-controller NodePort 10.101.67.244 \u0026lt;none\u0026gt; 80:30708/TCP,443:31969/TCP 116s service/ingress-nginx-controller-admission ClusterIP 10.106.3.163 \u0026lt;none\u0026gt; 443/TCP 116s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/ingress-nginx-controller 1/1 1 1 116s NAME DESIRED CURRENT READY AGE replicaset.apps/ingress-nginx-controller-cc8496874 1 1 1 116s NAME COMPLETIONS DURATION AGE job.batch/ingress-nginx-admission-create 1/1 12s 116s job.batch/ingress-nginx-admission-patch 1/1 13s 116s Using Ingress # The below example continues to build on the nginx-app Deployment and Service.\nstudent@minikube:~$ kubectl create ingress nginx-app-ingress --rule=\u0026#34;/=nginx-app:80\u0026#34; --rule=\u0026#34;/hello=newdeploy:8080\u0026#34; ingress.networking.k8s.io/nginx-app-ingress created We create a new Ingress resource with the name nginx-app-ingress:\nThe first rule routes traffic from the root / to our nginx-app Service on port 80. The second rule routes traffic from the URI /hello to a non existing newdeploy Service on port 8080. student@minikube:~$ kubectl describe ingress nginx-app-ingress Name: nginx-app-ingress Labels: \u0026lt;none\u0026gt; Namespace: default Address: 192.168.49.2 Default backend: default-http-backend:80 (\u0026lt;error: endpoints \u0026#34;default-http-backend\u0026#34; not found\u0026gt;) Rules: Host Path Backends ---- ---- -------- * / nginx-app:80 (172.17.0.4:80,172.17.0.5:80,172.17.0.6:80 + 2 more...) /hello newdeploy:8080 (\u0026lt;error: endpoints \u0026#34;newdeploy\u0026#34; not found\u0026gt;) Annotations: \u0026lt;none\u0026gt; Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Sync 3m10s (x2 over 3m11s) nginx-ingress-controller Scheduled for sync Notice that the backends or Pods for newdeploy are not found.\nBefore proceeding, update the /etc/hosts file to associate a domain with the IP address of our minikube container (which is running our K8s cluster). You can find the IP by running the minikube ip command. e.g. 192.168.42.2 nginx-app.demo\nNext, let\u0026rsquo;s test our Ingress resource:\nstudent@minikube:~$ kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE nginx-app-ingress nginx * 192.168.49.2 80 12m student@minikube:~$ curl nginx-app.demo \u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;Welcome to nginx!\u0026lt;/title\u0026gt; student@minikube:~$ curl nginx-app.demo/hello \u0026lt;html\u0026gt; \u0026lt;head\u0026gt;\u0026lt;title\u0026gt;503 Service Temporarily Unavailable\u0026lt;/title\u0026gt;\u0026lt;/head\u0026gt; We should fix the /hello URI by creating the newdeploy Deployment and Service:\nstudent@minikube:~$ kubectl create deployment newdeploy --image=gcr.io/google-samples/hello-app:2.0 deployment.apps/newdeploy created student@minikube:~$ kubectl expose deployment newdeploy --port=8080 service/newdeploy exposed student@minikube:~$ curl nginx-app.demo/hello Hello, world! Version: 2.0.0 Hostname: newdeploy-698574c958-kvnbc Configuring Ingress Rules # In the previous example, we\u0026rsquo;ve configured the nginx-app-ingress Ingress resource with the rules --rule=\u0026quot;/=nginx-app:80\u0026quot; --rule=\u0026quot;/hello=newdeploy:8080. Each Ingress Rules contains the following:\nAn optional host. If no host is specified, the rule applies to all inbound HTTP traffic. A list of paths, each path has its own backend. Paths can be exposed as regular expressions. The backend, which consists of either a service or a resource. You can configure a default backend for incoming traffic that doesn\u0026rsquo;t match any of the defined backends. The service backed relates to a Service while a resource backend refers to Cloud based object storage. We\u0026rsquo;ll focus on service backends. The Ingress pathType specifies how to deal with path requests:\nThe Exact pathType indicates that an exact match should occur: If the path is set to /foo and the request is /foo/, there is no match. The Prefix pathType indicates that the requested path should start with: If the path is set to /, any requested path will match. If the path is set to /foo, then /foo as well as /foo/ and /foo/bar will match. There are different Ingress Types:\nSingle Service: kubectl create ingress ingress-name --rule=\u0026quot;/hello=hello-service:80\u0026quot; Simple fanout: kubectl create ingress ingress-name --rule=\u0026quot;/hello=hello-service:80\u0026quot; --rule=\u0026quot;/goodbye=goodbye-service:80\u0026quot; Name-based Virtual Hosting: kubectl create ingress ingress-name --rule=\u0026quot;my.example.com/hello*=hello-service:80\u0026quot; --rule=\u0026quot;my.example.org/goodbye*=goodbye-service:80\u0026quot; Let\u0026rsquo;s cover this in an example:\nstudent@minikube:~$ kubectl create deploy foo --image=nginx deployment.apps/foo created student@minikube:~$ kubectl create deploy bar --image=httpd deployment.apps/bar created student@minikube:~$ kubectl expose deploy foo --port=80 service/foo exposed student@minikube:~$ kubectl expose deploy bar --port=80 service/bar exposed student@minikube:~$ kubectl create ingress multihost --rule=\u0026#34;foo.example.com/=foo:80\u0026#34; --rule=\u0026#34;bar.example.com/=bar:80\u0026#34; ingress.networking.k8s.io/multihost created Create the necessary /etc/hosts entries for foo.example.com and bar.example.com. Edit the multihost Ingress resource and set the pathType to Prefix for both backends\nstudent@minikube:~$ kubectl edit ingress multihost ingress.networking.k8s.io/multihost edited student@minikube:~$ kubectl get ingress multihost NAME CLASS HOSTS ADDRESS PORTS AGE multihost nginx foo.example.com,bar.example.com 192.168.49.2 80 75s student@minikube:~$ curl foo.example.com \u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;Welcome to nginx!\u0026lt;/title\u0026gt; student@minikube:~$ curl foo.example.com/lololol \u0026lt;html\u0026gt; \u0026lt;head\u0026gt;\u0026lt;title\u0026gt;404 Not Found\u0026lt;/title\u0026gt;\u0026lt;/head\u0026gt; student@minikube:~$ curl bar.example.com \u0026lt;html\u0026gt;\u0026lt;body\u0026gt;\u0026lt;h1\u0026gt;It works!\u0026lt;/h1\u0026gt;\u0026lt;/body\u0026gt;\u0026lt;/html\u0026gt; student@minikube:~$ curl bar.example.com/lololol \u0026lt;!DOCTYPE HTML PUBLIC \u0026#34;-//IETF//DTD HTML 2.0//EN\u0026#34;\u0026gt; \u0026lt;html\u0026gt;\u0026lt;head\u0026gt; \u0026lt;title\u0026gt;404 Not Found\u0026lt;/title\u0026gt; Network Policies # By default there are no restrictions to network traffic in Kubernetes: Pods can always communicate, even if they\u0026rsquo;re in other Name Spaces. We can limit this by using Network Policies, however, this needs to be supported by the network plugin. Remember that by default Kubernetes only offers basic network connectivity and this can be expanded with third party plugins.\nIf you don\u0026rsquo;t use a Network Policy, all traffic is allowed. If using a Network Policy and there\u0026rsquo;s no match, traffic is denied. Minikube doesn\u0026rsquo;t automatically start with a network plugin, so let\u0026rsquo;s restart minikube and configure it to use the Calico network plugin:\nstudent@minikube:~$ minikube stop ✋ Stopping node \u0026#34;minikube\u0026#34; ... 🛑 Powering off \u0026#34;minikube\u0026#34; via SSH ... 🛑 1 node stopped. student@minikube:~$ minikube delete 🔥 Deleting \u0026#34;minikube\u0026#34; in docker ... 🔥 Deleting container \u0026#34;minikube\u0026#34; ... 🔥 Removing /home/student/.minikube/machines/minikube ... 💀 Removed all traces of the \u0026#34;minikube\u0026#34; cluster. student@minikube:~$ minikube start --cni=calico 😄 minikube v1.25.2 on Ubuntu 18.04 (amd64) ✨ Automatically selected the docker driver. Other choices: ssh, none 👍 Starting control plane node minikube in cluster minikube 🚜 Pulling base image ... 🔥 Creating docker container (CPUs=2, Memory=2200MB) ... 🐳 Preparing Kubernetes v1.23.3 on Docker 20.10.12 ... ▪ kubelet.housekeeping-interval=5m ▪ Generating certificates and keys ... ▪ Booting up control plane ... ▪ Configuring RBAC rules ... 🔗 Configuring Calico (Container Networking Interface) ... 🔎 Verifying Kubernetes components... ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5 🌟 Enabled addons: storage-provisioner, default-storageclass 💡 kubectl not found. If you need it, try: \u0026#39;minikube kubectl -- get pods -A\u0026#39; 🏄 Done! kubectl is now configured to use \u0026#34;minikube\u0026#34; cluster and \u0026#34;default\u0026#34; namespace by default student@minikube:~$ kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-8594699699-r4rwl 1/1 Running 0 2m7s calico-node-8qzhj 1/1 Running 0 2m7s As with other Kubernetes resources, when defining a Pod- or NameSpace-based NetworkPolicy, a selector label is used to specify what traffic is allowed to and from the Pods that match the selector.\nThree different NetworkPolicy Identifiers can be used to match network traffic:\npodSelector: Allows access to a Pod with the corresponding selector label. namespaceSelector: Allows incoming traffic from namespaces with the matching selector label. ipBlock: Do not confuse with the verb to block - Specify a range of IP addresses that should get access. Here\u0026rsquo;s an example NetworkPolicy:\napiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: access-nginx spec: podSelector: matchLabels: app: nginx ingress: - from: - podSelector: matchLabels: access: \u0026#34;true\u0026#34; ... --- apiVersion: v1 kind: Pod metadata: name: nginx labels: app: nginx spec: containers: - name: nwp-nginx image: nginx:1.17 ... --- apiVersion: v1 kind: Pod metadata: name: busybox labels: app: sleepy spec: containers: - name: nwp-busybox image: busybox command: - sleep - \u0026#34;3600\u0026#34; The above NetworkPolicy can be understood as follows:\nApply the Network Policy to Pods that have the label app: nginx Allow incoming traffic from Pods that have the label access: \u0026quot;true\u0026quot; In other words, our nginx Pod will only accept traffic from Pods that have the access: \u0026quot;true\u0026quot; label set:\nstudent@minikube:~$ kubectl create -f ckad/nwpolicy-complete-example.yaml networkpolicy.networking.k8s.io/access-nginx created pod/nginx created pod/busybox created student@minikube:~$ kubectl get networkpolicy NAME POD-SELECTOR AGE access-nginx app=nginx 2m59s student@minikube:~$ kubectl describe networkpolicy Name: access-nginx Namespace: default Created on: 2021-12-01 18:12:12 +0000 UTC Labels: \u0026lt;none\u0026gt; Annotations: \u0026lt;none\u0026gt; Spec: PodSelector: app=nginx Allowing ingress traffic: To Port: \u0026lt;any\u0026gt; (traffic allowed to all ports) From: PodSelector: access=true Not affecting egress traffic Policy Types: Ingress student@minikube:~$ kubectl expose pod nginx --port=80 service/nginx exposed student@minikube:~$ kubectl exec -it busybox -- wget --spider --timeout=1 nginx Connecting to nginx (10.108.90.255:80) wget: download timed out command terminated with exit code 1 student@minikube:~$ kubectl label pod busybox access=true pod/busybox labeled student@minikube:~$ kubectl exec -it busybox -- wget --spider --timeout=1 nginx Connecting to nginx (10.108.90.255:80) remote file exists student@minikube:~$ ","date":"8 March 2025","externalUrl":null,"permalink":"/kubernetes-101-building-scalable-applications-networking/","section":"Blog","summary":"","title":"Kubernetes 101: Building Scalable Applications - Networking","type":"posts"},{"content":"","date":"8 March 2025","externalUrl":null,"permalink":"/tags/network-policies/","section":"Tags","summary":"","title":"Network Policies","type":"tags"},{"content":"","date":"8 March 2025","externalUrl":null,"permalink":"/tags/ports/","section":"Tags","summary":"","title":"Ports","type":"tags"},{"content":"","date":"8 March 2025","externalUrl":null,"permalink":"/tags/services/","section":"Tags","summary":"","title":"Services","type":"tags"},{"content":"","date":"28 November 2024","externalUrl":null,"permalink":"/tags/deployments/","section":"Tags","summary":"","title":"Deployments","type":"tags"},{"content":" Deployments # Deployments are the standard for running applications in Kubernetes, it protects Pods and will automatically restart them if anything goes wrong. Additionally, it offer features that add to the scalability and reliability of the application:\nScalability: Scaling the number of application instances to meet the demand. Updates and Update Strategy: Zero-downtime application updates We use the kubectl create deploy command to create a Deployment:\nstudent@minikube:~$ kubectl create deployment myweb --image=nginx --replicas=3 deployment.apps/myweb created student@minikube:~$ kubectl describe deploy myweb Name: myweb Namespace: default CreationTimestamp: Mon, 01 Nov 2021 09:08:57 +0000 Labels: app=myweb Annotations: deployment.kubernetes.io/revision: 1 Selector: app=myweb Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable StrategyType: RollingUpdate MinReadySeconds: 0 RollingUpdateStrategy: 25% max unavailable, 25% max surge Pod Template: Labels: app=myweb Containers: nginx: Image: nginx Port: \u0026lt;none\u0026gt; Host Port: \u0026lt;none\u0026gt; Environment: \u0026lt;none\u0026gt; Mounts: \u0026lt;none\u0026gt; Volumes: \u0026lt;none\u0026gt; Conditions: Type Status Reason ---- ------ ------ Available True MinimumReplicasAvailable Progressing True NewReplicaSetAvailable OldReplicaSets: \u0026lt;none\u0026gt; NewReplicaSet: myweb-8764bf4c8 (3/3 replicas created) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ScalingReplicaSet 2m29s deployment-controller Scaled up replica set myweb-8764bf4c8 to 3 student@minikube:~$ kubectl get all NAME READY STATUS RESTARTS AGE pod/myweb-8764bf4c8-6gxv8 1/1 Running 0 4m23s pod/myweb-8764bf4c8-6mvn8 1/1 Running 0 4m23s pod/myweb-8764bf4c8-q72nq 1/1 Running 0 4m23s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 36m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/myweb 3/3 3 3 4m23s NAME DESIRED CURRENT READY AGE replicaset.apps/myweb-8764bf4c8 3 3 3 4m23s We created the myweb deployment based on the nginx image with 3 replicas or desired Pods. Notice the Labels and Selector fields.\nThe Deployment created the ReplicaSet to ensure that a specified number of Pods are always running at any given time, and it created the Pods. Both the ReplicaSet and the Pods are managed by the Deployment.\nYou cannot manage Pods independently when they are part of a Deployment. When trying to delete a Pod, the Deployment kicks in and uses the ReplicaSet to make sure we have 3 running Pods:\nstudent@minikube:~$ kubectl delete pod myweb-8764bf4c8-6gxv8 kpod \u0026#34;myweb-8764bf4c8-6gxv8\u0026#34; deleted student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE myweb-8764bf4c8-6mvn8 1/1 Running 0 14m myweb-8764bf4c8-q72nq 1/1 Running 0 14m myweb-8764bf4c8-qf2vc 0/1 ContainerCreating 0 5s Deployment Scalability # Before Deployments existed, ReplicaSets were used to manage scalability. In the previous section we saw that our deployment created the necessary ReplicaSet: Manage ReplicaSets only through Deployments. We do not care about managing ReplicaSets individually.\nWe can use the kubectl scale deployment command to manually scale an existing deployment:\nkubectl scale deployment my-deployment --replicas=5\nstudent@minikube:~$ kubectl scale deployment myweb --replicas=5 deployment.apps/myweb scaled student@minikube:~$ kubectl describe deploy myweb | grep -i replicas Replicas: 5 desired | 5 updated | 5 total | 5 available | 0 unavailable student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE myweb-8764bf4c8-44zxq 0/1 ContainerCreating 0 3s myweb-8764bf4c8-6mvn8 1/1 Running 0 36m myweb-8764bf4c8-7dpnx 0/1 ContainerCreating 0 3s myweb-8764bf4c8-q72nq 1/1 Running 0 36m myweb-8764bf4c8-qf2vc 1/1 Running 0 22m Additionally, there\u0026rsquo;s the kubectl edit deployment command which opens a text-editor for you, similar to systemctl edit for editing Systemd Unit files. This command, however, does not allow you to modify every single setting of a deployment.\nIn the below example I changed the deployment namespace and replicas:\nstudent@minikube:~$ kubectl edit deploy myweb A copy of your changes has been stored to \u0026#34;/tmp/kubectl-edit-3283969971.yaml\u0026#34; error: the namespace from the provided object \u0026#34;secret\u0026#34; does not match the namespace \u0026#34;default\u0026#34;. You must pass \u0026#39;--namespace=secret\u0026#39; to perform this operation. As you can see, Kubernetes isn\u0026rsquo;t happy about changing the namespace.\nDeployment Updates # Deployments allow for zero-downtime application updates.\nWhen an update is applied, a new ReplicaSet is created with the new properties: Pods with the new properties are started in the new ReplicaSet. After updating, the old ReplicaSet is no longer used and may be deleted. Or, you can keep it around for rolling-back. The deployment.spec.revisionHistoryLimit is set to keep the last 10 ReplicaSets.\nThe deployment.spec.strategy.type property defines how to handle updates:\nRollingUpdate: The default value. Replaces old Pods with new Pods in such a way to ensure the application remains available to users. Recreate: Kill all existing Pods before creating new ones. The application will be down. More on the this later\u0026hellip; Let\u0026rsquo;s perform a rolling update of Nginx using the kubectl set command. The command only accepts a limited amount of arguments.\nstudent@minikube:~$ kubectl create deploy mynginx --image=nginx:1.14 deployment.apps/mynginx created student@minikube:~$ kubectl describe deploy mynginx Name: mynginx Namespace: default CreationTimestamp: Sat, 02 Apr 2022 10:29:52 +0000 Labels: app=mynginx Annotations: deployment.kubernetes.io/revision: 1 Selector: app=mynginx Replicas: 1 desired | 1 updated | 1 total | 0 available | 1 unavailable StrategyType: RollingUpdate MinReadySeconds: 0 RollingUpdateStrategy: 25% max unavailable, 25% max surge Pod Template: Labels: app=mynginx Containers: nginx: Image: nginx:1.14 student@minikube:~$ kubectl get all --selector app=mynginx NAME READY STATUS RESTARTS AGE pod/mynginx-6b9d85f696-w4wpt 1/1 Running 0 64s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mynginx 1/1 1 1 64s NAME DESIRED CURRENT READY AGE replicaset.apps/mynginx-6b9d85f696 1 1 1 64s Notice the Image field in the output of the kubectl describe command, the default StrategyType, as well as how the middle part of the Pod name matches the suffix of the ReplicaSet name: pod/mynginx-6b9d85f696-w4wpt =\u0026gt; replicaset.apps/mynginx-6b9d85f6960. We can conclude that this Pod belongs to that ReplicaSet.\nNow, update the image version to 1.17. The kubectl set command only accepts a limited amount of arguments.\nstudent@minikube:~$ kubectl set env image resources selector serviceaccount subject student@minikube:~$ kubectl set image deploy mynginx nginx=nginx:1.17 deployment.apps/mynginx image updated student@minikube:~$ kubectl get all --selector app=mynginx NAME READY STATUS RESTARTS AGE pod/mynginx-6b9d85f696-w4wpt 1/1 Running 0 7m4s pod/mynginx-6d9cd8f877-g4dkv 0/1 ContainerCreating 0 8s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mynginx 1/1 1 1 7m4s NAME DESIRED CURRENT READY AGE replicaset.apps/mynginx-6b9d85f696 1 1 1 7m4s replicaset.apps/mynginx-6d9cd8f877 1 1 0 9s We see that our old ReplicaSet and Pod are still there, our application is still available, while a new Pod with the new Nginx image is being created. Once the new Pod is running, the old Pod will be deleted but the old (empty) ReplicaSet will still be there:\nstudent@minikube:~$ kubectl get all --selector app=mynginx NAME READY STATUS RESTARTS AGE pod/mynginx-6d9cd8f877-g4dkv 1/1 Running 0 2m4s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mynginx 1/1 1 1 9m NAME DESIRED CURRENT READY AGE replicaset.apps/mynginx-6b9d85f696 0 0 0 9m replicaset.apps/mynginx-6d9cd8f877 1 1 1 2m5s The rolling update is complete and the old ReplicaSet is still available in case we need to roll back (covered later on in this article).\nLabels, Selectors, and Annotations # Labels are key:value pairs that are defined in resources like Pods, Deployments and Services. They are either set automatically or can be set manually by an administrator. Each label key that is attached to a single object resource must be unique, though different objects can have the same label key:value pairs. This allows us to group objects, or map a specific structure onto objects, and query only the objects with a specific label.\nIf we look back at our previous deployment, we can see that each object in the deployment has the app=mynginx label set:\nstudent@minikube:~$ kubectl describe pod mynginx-6d9cd8f877-g4dkv | grep Labels: Labels: app=mynginx student@minikube:~$ kubectl describe rs mynginx | grep Labels: Labels: app=mynginx student@minikube:~$ kubectl describe deploy mynginx | grep Labels: Labels: app=mynginx So using a label selector, we can target the related objects of a specific application.\ne.g.:\nstudent@minikube:~$ kubectl get all --selector app=mynginx NAME READY STATUS RESTARTS AGE pod/mynginx-6d9cd8f877-g4dkv 1/1 Running 0 29m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mynginx 1/1 1 1 36m NAME DESIRED CURRENT READY AGE replicaset.apps/mynginx-6b9d85f696 0 0 0 36m replicaset.apps/mynginx-6d9cd8f877 1 1 1 29m Our kubectl create deployment command automatically set the app=appname label, where appname is the name of the deployment.\nExample:\nstudent@minikube:~$ kubectl create deploy mylabel --image=nginx deployment.apps/mylabel created student@minikube:~$ kubectl label deploy mylabel state=demo deployment.apps/mylabel labeled student@minikube:~$ kubectl get deploy --show-labels NAME READY UP-TO-DATE AVAILABLE AGE LABELS mylabel 1/1 1 1 45s app=mylabel,state=demo student@minikube:~$ kubectl get deploy --selector state=demo NAME READY UP-TO-DATE AVAILABLE AGE mylabel 1/1 1 1 70s Notice that while we\u0026rsquo;ve given the deployment mylabel a new label, this new label is not inherited by the resources or objects created by the deployment:\nstudent@minikube:~$ kubectl get all --show-labels NAME READY STATUS RESTARTS AGE LABELS pod/mylabel-566dc5f574-ctkqg 1/1 Running 0 7m28s app=mylabel,pod-template-hash=566dc5f574 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE LABELS service/kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 168m component=apiserver,provider=kubernetes NAME READY UP-TO-DATE AVAILABLE AGE LABELS deployment.apps/mylabel 1/1 1 1 7m28s app=mylabel,state=demo NAME DESIRED CURRENT READY AGE LABELS replicaset.apps/mylabel-566dc5f574 1 1 1 7m28s app=mylabel,pod-template-hash=566dc5f574 student@minikube:~$ kubectl get all --selector state=demo NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mylabel 1/1 1 1 8m58s We can also remove a label. Let\u0026rsquo;s remove the label with the key app from the Pod mylabel-566dc5f574-ctkqg:\nstudent@minikube:~$ kubectl label pod mylabel-566dc5f574-ctkqg app- pod/mylabel-566dc5f574-ctkqg unlabeled student@minikube:~$ kubectl get all NAME READY STATUS RESTARTS AGE pod/mylabel-566dc5f574-ctkqg 1/1 Running 0 12m pod/mylabel-566dc5f574-pxkdz 0/1 ContainerCreating 0 5s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 174m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mylabel 0/1 1 0 12m NAME DESIRED CURRENT READY AGE replicaset.apps/mylabel-566dc5f574 1 1 0 12m student@minikube:~$ kubectl get all --selector app=mylabel NAME READY STATUS RESTARTS AGE pod/mylabel-566dc5f574-pxkdz 1/1 Running 0 3m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mylabel 1/1 1 1 15m NAME DESIRED CURRENT READY AGE replicaset.apps/mylabel-566dc5f574 1 1 1 15m Our deployment could no longer find the Pod which is supposed to have the app=mylabel label, so it created a new Pod: mylabel-566dc5f574-pxkdz.\nSince the Pod with the removed label is no longer managed by our deployment, we can delete it without our deployment (or rather ReplicaSet) recreating it.\nAnnotations can\u0026rsquo;t be used in queries, but are useful to provide detailed non-identifying metadata in an object: maintainer, author, license, \u0026hellip;\nUpdate Strategy # When a Deployment changes, the Pods are immediately updated according to the Update Strategy:\nRollingUpdate: Updates Pods one at a time to guarantee availability of the application. Recreate: All Pods are killed and new Pods are created. This leads to temporary unavailability of the application which can be useful when different versions of an application cannot run simultaneously (e.g. a database). The task of the Deployment is to ensure that enough Pods are running at all times. When a change is made, the changed version is deployed in a new ReplicaSet. The old ReplicaSet is scaled to 0 (deactivated) once the update was confirmed as successful. We can use kubectl rollout history to get details about recent transactions, and kubectl rollout undo to undo a previous change.\nThe RollingUpdate options guarantee a certain minimal and maximum number of Pods to be always available:\nmaxUnavailable: Determines the maximum number of Pods that are upgraded at the same time. maxSurge: The number of Pods that can run beyond the desired number of Pods specified in the ReplicaSet to guarantee minimal availability. student@minikube:~$ kubectl get deploy mylabel -o yaml ... spec: progressDeadlineSeconds: 600 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: mylabel strategy: rollingUpdate: maxSurge: 25% maxUnavailable: 25% type: RollingUpdate ... Deployment History # At this point we know that Deployment updates create a new ReplicaSet with new properties, the old ReplicaSet is kept but is scaled down to 0 Pods. Since the old ReplicaSet is kept around, we can easily undo a change. We can use kubectl rollout history to get details about recent roll outs, and kubectl rollout undo to undo a previous change.\nLet\u0026rsquo;s start by updating our mylabel deployment. We\u0026rsquo;ll give all the Pods a new environment variable: foo=bar:\nkubectl set env deploy mylabel foo=bar deployment.apps/mylabel env updated student@minikube:~$ kubectl rollout history deploy mylabel deployment.apps/mylabel REVISION CHANGE-CAUSE 1 \u0026lt;none\u0026gt; 2 \u0026lt;none\u0026gt; student@minikube:~$ kubectl rollout history deploy mylabel --revision=1 deployment.apps/mylabel with revision #1 Pod Template: Labels:\tapp=mylabel pod-template-hash=566dc5f574 Containers: nginx: Image:\tnginx Port:\t\u0026lt;none\u0026gt; Host Port:\t\u0026lt;none\u0026gt; Environment:\t\u0026lt;none\u0026gt; Mounts:\t\u0026lt;none\u0026gt; Volumes:\t\u0026lt;none\u0026gt; student@minikube:~$ kubectl rollout history deploy mylabel --revision=2 deployment.apps/mylabel with revision #2 Pod Template: Labels:\tapp=mylabel pod-template-hash=57f55bcb47 Containers: nginx: Image:\tnginx Port:\t\u0026lt;none\u0026gt; Host Port:\t\u0026lt;none\u0026gt; Environment: foo:\tbar Mounts:\t\u0026lt;none\u0026gt; Volumes:\t\u0026lt;none\u0026gt; We can see that we added the environment variable in revision 2. So let\u0026rsquo;s roll back revision 1:\nstudent@minikube:~$ kubectl rollout undo deploy mylabel --to-revision=1 deployment.apps/mylabel rolled back Deployment Alternatives # There are two additional Deployments alternatives:\nStatefulSets: the workload API object used to manage stateful applications. We\u0026rsquo;ll cover these once we know more about Networking and Storage. DaemonSet: ensures that all (or some) Nodes run a copy of a Pod (1 Pod, no replicas). As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created. A simple use case for a DaemonSet is for example the need to run some sort of Agent on every worker-node.\nThe YAML code for DaemonSets needs to be created from scratch, you can\u0026rsquo;t use kubectl create to generate the YAML :( Example YAML code:\napiVersion: apps/v1 kind: DaemonSet metadata: name: nginxdaemon namespace: default labels: k8s-app: nginxdaemon spec: selector: matchLabels: name: nginxdaemon template: metadata: labels: name: nginxdaemon spec: containers: - name: nginx image: nginx student@minikube:~$ kubectl create -f daemon.yaml daemonset.apps/nginxdaemon created student@minikube:~$ kubectl get ds,pods NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/nginxdaemon 1 1 1 1 1 \u0026lt;none\u0026gt; 13s NAME READY STATUS RESTARTS AGE pod/nginxdaemon-5nn27 1/1 Running 0 13s ","date":"28 November 2024","externalUrl":null,"permalink":"/kubernetes-101-building-scalable-applications-deployments/","section":"Blog","summary":"","title":"Kubernetes 101: Building Scalable Applications - Deployments","type":"posts"},{"content":"","date":"28 November 2024","externalUrl":null,"permalink":"/tags/labels/","section":"Tags","summary":"","title":"Labels","type":"tags"},{"content":"","date":"28 November 2024","externalUrl":null,"permalink":"/tags/selectors/","section":"Tags","summary":"","title":"Selectors","type":"tags"},{"content":"","date":"28 November 2024","externalUrl":null,"permalink":"/tags/updatestrategy/","section":"Tags","summary":"","title":"UpdateStrategy","type":"tags"},{"content":" Managing Basic Pod Features # Deployments are the standard for running applications in Kubernetes. For the sake of getting familiar with Kubernetes and understanding the essentials, we\u0026rsquo;ll be creating and running native Pods.\nUnderstanding Pods # A Pod is an abstraction of a server which can run multiple containers within a single namespace, exposed by a single IP address. The Pod is the smallest entity that can be created and managed by Kubernetes: Kubernetes does not manage containers, it manages Pods.\nManaging Pods with kubectl # Typically managed Pods are started through a Deployment resource.\nNaked Pods are started using the kubectl run option: kubectl run mynginx --image=nginx\nNaked Pods cannot be scaled, are not rescheduled in case of failure, cannot be replaced automatically and can\u0026rsquo;t have rolling updates.\nkubectl run -h: Show all options for creating a Pod. kubectl run mynginx --image=nginx: Start a Pod with the name mynginx from the nginx Dockerhub image. kubectl get pods: Show the parameters of all Pods kubectl get pods mynginx: Show the parameters of a specific Pod kubectl get pods mynginx -o yaml: Show the output in YAML format. kubectl describe pods: Show all details about all pods kubectl describe pods mynginx: Show all details about a specific Pod YAML # YAML is a human-readable data-serialization language which uses indentation to identify relations.\nBasic YAML Manifest Ingredients # All of the YAML manifest ingredients are defined in the API. You can use kubectl explain to get more information about the YAML fields or properties:\nstudent@minikube:~$ kubectl explain pods KIND: Pod VERSION: v1 DESCRIPTION: Pod is a collection of containers that can run on a host. This resource is created by clients and scheduled onto hosts. FIELDS: apiVersion\t\u0026lt;string\u0026gt; APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources kind\t\u0026lt;string\u0026gt; Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds metadata\t\u0026lt;Object\u0026gt; Standard object\u0026#39;s metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata spec\t\u0026lt;Object\u0026gt; Specification of the desired behavior of the pod. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status status\t\u0026lt;Object\u0026gt; Most recently observed status of the pod. This data may not be up to date. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status student@minikube:~$ kubectl explain pods.spec ... student@minikube:~$ kubectl explain pods.spec.containers ... The kubectl explain pods.spec.containers command shows us that the container spec has multiple fields of which the below are the most important ones:\nFIELDS: name \u0026lt;string\u0026gt; -required- Name of the container specified as a DNS_LABEL. image \u0026lt;string\u0026gt; Docker image name. command \u0026lt;[]string\u0026gt; Entrypoint array. Not executed within a shell. The docker image\u0026#39;s ENTRYPOINT is used if this is not provided. args \u0026lt;[]string\u0026gt; Arguments to the entrypoint. The docker image\u0026#39;s CMD is used if this is not provided env \u0026lt;[]Object\u0026gt; List of environment variables to set in the container. Cannot be updated. If you have a YAML file with a Pod spec you can create a Pod from it:\nstudent@minikube:~$ cat busybox.yaml apiVersion: v1 kind: Pod metadata: name: busybox2 namespace: default spec: containers: - name: busy image: busybox command: - sleep - \u0026#34;3600\u0026#34; student@minikube:~$ kubectl create -f busybox.yaml pod/busybox2 created student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE busybox2 0/1 ContainerCreating 0 7s Similarly you can delete or update (apply Spec changes) the Pod using the same YAML file:\nstudent@minikube:~$ kubectl delete -f busybox.yaml pod \u0026#34;busybox2\u0026#34; deleted student@minikube:~$ kubectl apply -f busybox.yaml pod/busybox2 created student@minikube:~$ kubectl apply -f busybox.yaml pod/busybox2 unchanged Generating YAML files # By using YAML files we use Kubernetes in a declarative way where the files are typically stored in a git repository and which fits well into a DevOps strategy. The imperative way of working with Kubernetes is where you create everything from the command line.\nWe can write YAML files but we should generate them instead and modify it to suit our specific needs:\nkubectl run mynginx --image=nginx --dry-run=client -o yaml \u0026gt; mynginx.yaml\nThe --dry-run option prevents Kubernetes from actually running the Pod.\nUnderstanding and Configuring Multi-Container Pods # The one-container Pod is the standard, they are easier to build and maintain. Typically, to create applications that consists of multiple containers, micro-services should be used. In a microservice, different independently managed Pods are connected by resources provided by Kubernetes.\nThere are some use cases where you might want to run multiple containers in a single Pod:\nSidecar container: A container that enhances the primary application, for example logging. Ambassador container: A container that represents the primary container to the outside world, for example a proxy. Adapter container: Used to adopt the traffic or data pattern to match the traffic or data pattern in other applications in the cluster. These containers are not defined by specific Pod properties, you won\u0026rsquo;t find information on their specs in kubectl explain pod.spec.\nSidecar Containers # A sidecar container is providing additional functionality to the main container where it makes no sense to run this functionality in a separate Pod. The essence is that the main container and sidecar container have access to shared resources in order to exchange information.\ne.g. Istio service mesh injects sidecar containers in Pods to enable traffic management.\nHere\u0026rsquo;s a basic example of a multi-container Pod:\nstudent@minikube:~$ cat sidecar.yaml kind: Pod apiVersion: v1 metadata: name: sidecar-pod spec: volumes: - name: logs emptyDir: {} containers: - name: main image: busybox command: [\u0026#34;/bin/sh\u0026#34;] args: [\u0026#34;-c\u0026#34;, \u0026#34;while true; do date \u0026gt;\u0026gt; /var/log/date.txt; sleep 10;done\u0026#34;] volumeMounts: - name: logs mountPath: /var/log - name: sidecar image: centos/httpd ports: - containerPort: 80 volumeMounts: - name: logs mountPath: /var/www/html The shared resource in the above example is the volume with the name logs and the emptyDir: {} property. An emptyDir volume is initially empty and can be mounted at different paths in different containers as we can see in the above YAML by looking at the container volumeMounts.\nThe main container writes the current date and time to /var/log/date.txt every 10 seconds, while the sidecar container will be able to read and present the file to a user since it has the same volume mounted albeit on a different path from the container perspective.\nLet\u0026rsquo;s create the Pod, open a shell session in the sidecar container and run cURL to check the output created by the main container:\nstudent@minikube:~$ kubectl create -f sidecar.yaml pod/sidecar-pod created student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE sidecar-pod 2/2 Running 0 10s student@minikube:~$ kubectl exec -it sidecar-pod -c sidecar -- /bin/bash [root@sidecar-pod /]# yum install curl -y .... [root@sidecar-pod /]# curl http://localhost/date.txt .... Managing Init Containers # An init container is an additional container in a Pod that needs to complete a task before the \u0026ldquo;regular\u0026rdquo; container is started. As long as the init container hasn\u0026rsquo;t completed its job, the regular container is not started.\nHave a look at this official example YAML file fo init containers. We\u0026rsquo;ll work with a more simplified version here:\napiVersion: v1 kind: Pod metadata: name: init-demo spec: containers: - name: nginx image: nginx initContainers: - name: init-box image: busybox command: - sleep - \u0026#34;3600\u0026#34; In the above example our init-box container will sleep for 1 hour and only once the sleep command finishes our nginx container will spin up. We can see our Pod is in the Init status:\nstudent@minikube:~$ kubectl create -f init-demo.yaml pod/init-demo created student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE init-demo 0/1 Init:0/1 0 4s We can use the describe command to get more information about the Pod:\nstudent@minikube:~$ kubectl describe pod init-demo ... Init Containers: init-box: Container ID: docker://c23ec32c3ba19d43417c730117b6319b0c57d6c8938c76ae641b1afad0e08c11 Image: busybox Image ID: docker-pullable://busybox@sha256:caa382c432891547782ce7140fb3b7304613d3b0438834dce1cad68896ab110a Port: \u0026lt;none\u0026gt; Host Port: \u0026lt;none\u0026gt; Command: sleep 3600 State: Running ... Containers: nginx: Container ID: Image: nginx Image ID: Port: \u0026lt;none\u0026gt; Host Port: \u0026lt;none\u0026gt; State: Waiting Reason: PodInitializing ... The Events section in the output of the describe command shows us what containers have been started:\nEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 78s default-scheduler Successfully assigned default/init-demo to minikube Normal Pulling 77s kubelet Pulling image \u0026#34;busybox\u0026#34; Normal Pulled 65s kubelet Successfully pulled image \u0026#34;busybox\u0026#34; in 11.886228471s Normal Created 65s kubelet Created container init-box Normal Started 64s kubelet Started container init-box Using NameSpaces # Kubernetes leverages Linux kernel-level resource isolation: NameSpaces. Different NameSpaces can be used to strictly separate between customer resources and to apply different security-related settings such as Role-Based Access Control and Quotas.\nLet\u0026rsquo;s demonstrate this the imparative way:\n# Show all available namespaces student@minikube:~$ kubectl get ns NAME STATUS AGE default Active 14d kube-node-lease Active 14d kube-public Active 14d kube-system Active 14d # Show all resources per namespace student@minikube:~$ kubectl get all -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system pod/coredns-64897985d-sj5lw 1/1 Running 3 (108s ago) 14d ... # Create a new namespace student@minikube:~$ kubectl create ns secret namespace/secret created # Start a new Pod in the new namespace student@minikube:~$ kubectl run secretnginx --image=nginx -n secret pod/secretnginx created student@minikube:~$ kubectl get pods No resources found in default namespace. # List all Pods in the secret namespace student@minikube:~$ kubectl get pods -n secret NAME READY STATUS RESTARTS AGE secretnginx 0/1 ContainerCreating 0 10s We can do the same thing the declarative way by defining namespace under the Pod metadata:\napiVersion: v1 kind: Pod metadata: name: busyboxPod namespace: secret Check properties of the namespace using the describe command:\nstudent@minikube:~$ kubectl describe ns secret Name: secret Labels: kubernetes.io/metadata.name=secret Annotations: \u0026lt;none\u0026gt; Status: Active No resource quota. No LimitRange resource. Lastly, let\u0026rsquo;s declaratively combine the creation of a namespace and a pod inside the same namespace:\nstudent@minikube:~$ kubectl create ns production --dry-run=client -o yaml \u0026gt; nginx_prod.yml student@minikube:~$ cat nginx_prod.yml apiVersion: v1 kind: Namespace metadata: creationTimestamp: null name: production spec: {} status: {} Notice that kind is Namespace.\nNow, we add the Pod to the same namespace inside the same Yaml file:\nstudent@minikube:~$ kubectl run nginx-prod -n production --image=nginx --dry-run=client -o yaml \u0026gt;\u0026gt; nginx_prod.yml student@minikube:~$ cat nginx_prod.yml apiVersion: v1 kind: Namespace metadata: creationTimestamp: null name: production spec: {} status: {} apiVersion: v1 kind: Pod metadata: creationTimestamp: null labels: run: nginx-prod name: nginx-prod namespace: production spec: containers: - image: nginx name: nginx-prod resources: {} dnsPolicy: ClusterFirst restartPolicy: Always status: {} We should modify the Yaml file in such a way that it\u0026rsquo;s clear we\u0026rsquo;re dealing with 2 Yaml list items in a single file. We\u0026rsquo;ll add the --- lines to indicate the start of a new list item:\nstudent@minikube:~$ cat nginx_prod.yml --- apiVersion: v1 kind: Namespace metadata: creationTimestamp: null name: production spec: {} status: {} --- apiVersion: v1 kind: Pod metadata: creationTimestamp: null labels: run: nginx-prod name: nginx-prod namespace: production spec: containers: - image: nginx name: nginx-prod resources: {} dnsPolicy: ClusterFirst restartPolicy: Always status: {} \u0026hellip; and we can now create the actual resources from the Yaml file:\nstudent@minikube:~$ kubectl create -f nginx_prod.yml namespace/production created pod/nginx-prod created student@minikube:~$ kubectl get all -n production NAME READY STATUS RESTARTS AGE pod/nginx-prod 0/1 ContainerCreating 0 15s student@minikube:~$ Managing Advanced Pod Features # Exploring Pod State # kubectl describe pod podname is a human-readable way to see all Pod parameters and settings as currently stored in the etcd database. You can use the offical documentation for more information about these settings and parameters.\nWhile we can describe the Pod externally, we can also connect to the Pod and run commands on the primary container in the Pod:\nConnect using kubectl exec -it podname -- sh Disconnect by executing the exit command. (or CTR+P CTRL+Q if the shell is running as process ID 1) student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE mynginx 1/1 Running 0 44s student@minikube:~$ kubectl get pods mynginx -o json | less ... student@minikube:~$ kubectl get pods mynginx -o yaml | less ... student@minikube:~$ kubectl describe pods mynginx ... student@minikube:~$ kubectl exec -it mynginx -- sh # pwd / # ps aux sh: 1: ps: not found # cd /proc # ls 1 acpi cmdline\tdiskstats filesystems irq\tkmsg\tlocks mounts\tsched_debug softirqs sysvipc\tversion 34 asound consoles dma\tfs\tkallsyms kpagecgroup mdstat mtrr\tschedstat stat\tthread-self version_signature 35 buddyinfo cpuinfo\tdriver interrupts kcore kpagecount meminfo net\tscsi\tswaps\ttimer_list vmallocinfo 53 bus crypto\texecdomains iomem\tkey-users kpageflags misc pagetypeinfo self\tsys\ttty\tvmstat 60 cgroups devices\tfb\tioports\tkeys loadavg\tmodules partitions\tslabinfo sysrq-trigger uptime\tzoneinfo # cat 1/cmdline nginx: master process nginx -g daemon off; # cat 53/cmdline sh # cat 35/cmdline nginx: worker process # exit student@minikube:~$ Most containers run minimal images where not all commands may be available, in the above example the ps command is not available. In this case we can make advantage of the proc pseudo filesystem.\nUsing Pod Logs # The Pod entrypoint application does not connect to any STDOUT, instead, application output is sent to the Kubernetes cluster. We can use kubectl logs to see this output and help us in troubleshooting:\nstudent@minikube:~$ kubectl run mydb --image=mariadb pod/mydb created student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE mydb 0/1 ContainerCreating 0 5s ... student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE mydb 0/1 CrashLoopBackOff 1 (15s ago) 76s student@minikube:~$ kubectl describe pod mydb ... State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 1 ... student@minikube:~$ kubectl logs mydb [ERROR] [Entrypoint]: Database is uninitialized and password option is not specified You need to specify one of MARIADB_ROOT_PASSWORD, MARIADB_ALLOW_EMPTY_ROOT_PASSWORD and MARIADB_RANDOM_ROOT_PASSWORD Looking at the log output, we needed to specify one or more specific environment variables. Let\u0026rsquo;s fix this, but since we can\u0026rsquo;t update a Pod (only deployments which we\u0026rsquo;ll see later) we need to delete our Pod first:\nstudent@minikube:~$ kubectl delete pod mydb pod \u0026#34;mydb\u0026#34; deleted student@minikube:~$ kubectl run mydb --image=mariadb --env=\u0026#34;MARIADB_ROOT_PASSWORD=myrootpassword\u0026#34; pod/mydb created student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE mydb 1/1 Running 0 40s student@minikube:~$ kubectl logs mydb [Note] mariadbd: ready for connections. Version: \u0026#39;10.7.3-MariaDB-1:10.7.3+maria~focal\u0026#39; socket: \u0026#39;/run/mysqld/mysqld.sock\u0026#39; port: 3306 mariadb.org binary distribution Port Forwarding # A simple way of accessing a Pod is by using Port Forwarding: Expose a port on the host running the Pod that forwards to the Pod. This is useful for testing Pod accessibility on a specific cluster node but isn\u0026rsquo;t used to expose the Pod to external users. Regular user access to applications in the Pod is provided via Services and Ingress.\nWhen you run kubectl get pods -o wide or kubectl describe pod podname you\u0026rsquo;ll see the Pod has an IP address. This IP address is accessible only from within the cluster, you cannot use it to address the Pod from outside the cluster.\nstudent@minikube:~$ kubectl get pods mynginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mynginx 1/1 Running 0 63m 172.17.0.3 minikube \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; student@minikube:~$ ping 172.17.0.3 PING 172.17.0.3 (172.17.0.3) 56(84) bytes of data. ^C --- 172.17.0.3 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2053ms student@minikube:~$ curl 172.17.0.3 curl: (7) Failed to connect to 172.17.0.3 port 80: No route to host So if we need to test network accessibility to our Pod, we use Port Forwarding:\nstudent@minikube:~$ kubectl port-forward mynginx 8080:80 \u0026amp; [1] 19855 student@minikube:~$ Forwarding from 127.0.0.1:8080 -\u0026gt; 80 Forwarding from [::1]:8080 -\u0026gt; 80 student@minikube:~$ This command starts a port forwarding process in the foreground, so we add the \u0026amp; at the end of the command to start it in the background.\nstudent@minikube:~$ curl localhost:8080 Handling connection for 8080 \u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;Welcome to nginx!\u0026lt;/title\u0026gt; To stop port forwarding, we bring the process back to the foreground and stop it using CTRL+C:\nstudent@minikube:~$ fg minikube kubectl -- port-forward mynginx 8080:80 ^C student@minikube:~$ Configuring securityContext # A securityContext defines privileges and access control settings for a Pod and/or container, and includes:\nDiscretionary Access Control SELinux or AppArmor Running as privileged or unprivileged user AllowPrivilegeEscalation to control if a process can gain more privileges than its parent process kubectl explain can give you a complete overview.\nLet\u0026rsquo;s work with examples:\nstudent@minikube:~$ kubectl explain pod.spec.securityContext ... student@minikube:~$ kubectl explain pod.spec.containers.securityContext ... student@minikube:~/ckad$ cat securitycontextdemo2.yaml apiVersion: v1 kind: Pod metadata: name: security-context-demo spec: securityContext: runAsUser: 1000 runAsGroup: 3000 fsGroup: 2000 volumes: - name: sec-ctx-vol emptyDir: {} containers: - name: sec-ctx-demo image: busybox command: [ \u0026#34;sh\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;sleep 1h\u0026#34; ] volumeMounts: - name: sec-ctx-vol mountPath: /data/demo securityContext: allowPrivilegeEscalation: false student@minikube:~/ckad$ kubectl create -f securitycontextdemo2.yaml pod/security-context-demo created student@minikube:~/ckad$ kubectl get pods security-context-demo -o yaml ... spec: containers: - command: - sh - -c - sleep 1h image: busybox imagePullPolicy: Always name: sec-ctx-demo resources: {} securityContext: allowPrivilegeEscalation: false ... student@minikube:~/ckad$ kubectl exec -it security-context-demo -- sh / $ cd data/demo /data/demo $ echo \u0026#34;Hello\u0026#34; \u0026gt; test /data/demo $ ls -l total 4 -rw-r--r-- 1 1000 2000 6 Mar 24 17:05 test /data/demo $ id uid=1000 gid=3000 groups=2000 When we create a new file in the Pods primary container, we see that the owner of the file is id 1000 (runAsUser) and group owner is 2000 (fsGroup) as specified in the Yaml securityContext. The id command reveals our runAsUser ID, our primary group id 3000 and our secondary group id 2000.\nManaging Jobs # Pods are the essence of Kubernetes, when your Pod goes down then Kubernetes will start a new Pod. In that sense, Pods are normally created to run forever. There can be cases where you want a Pod to execute a one-shot task, like backup jobs, a calculation or batch processing. This is were you can use Jobs: The Pod will run until it finishes its task then stops.\nWe can set ttlSecondsAfterFinished to clean up completed Jobs automatically so that we don\u0026rsquo;t keep both the Job and the Pod (created by the Job) around forever.\nThere are 3 different Job types specified by the completion and parallelism parameters:\nNon-parallel Jobs: 1 Job - 1 Pod completions=X paralellism=1 Parallel Jobs with a fixed completion count: the Job is completed after successfully running as many times as specified by jobs.spec.completions. The number of parallel or concurrent Pods that are started by the Job are specified by jobs.spec.parallelism. completions=X paralellism=Y Parallel Jobs with a work queue: Multiple Jobs are started, when one completes the Job is done. completions=1 parallelism=X Here\u0026rsquo;s an example:\nstudent@minikube:~$ kubectl create job onejob --image=busybox --dry-run=client -o yaml -- date \u0026gt; onejob.yml student@minikube:~$ cat onejob.yml apiVersion: batch/v1 kind: Job metadata: creationTimestamp: null name: onejob spec: template: metadata: creationTimestamp: null spec: containers: - command: - date image: busybox name: onejob resources: {} restartPolicy: Never status: {} Notice that kind is Job and that restartPolicy is set to Never. In this example the container just executes the date command and then is done.\nstudent@minikube:~$ kubectl create -f onejob.yml job.batch/onejob created student@minikube:~$ kubectl get jobs NAME COMPLETIONS DURATION AGE onejob 0/1 4s 4s student@minikube:~$ kubectl get jobs,pods NAME COMPLETIONS DURATION AGE job.batch/onejob 0/1 7s 7s NAME READY STATUS RESTARTS AGE pod/onejob-zjgd9 0/1 ContainerCreating 0 7s Once the Job is done, the Job COMPLETIONS and Pod STATUS is updated:\nstudent@minikube:~$ kubectl get jobs,pods NAME COMPLETIONS DURATION AGE job.batch/onejob 1/1 10s 41s NAME READY STATUS RESTARTS AGE pod/onejob-zjgd9 0/1 Completed 0 41s student@minikube:~$ kubectl delete -f onejob.yml job.batch \u0026#34;onejob\u0026#34; deleted Now let\u0026rsquo;s create a parallel Job:\nstudent@minikube:~$ kubectl create job paralleljob --image=busybox --dry-run=client -o yaml -- sleep 5 \u0026gt; paralleljob.yml student@minikube:~$ cat paralleljob.yml apiVersion: batch/v1 kind: Job metadata: creationTimestamp: null name: paralleljob spec: completions: 6 parallelism: 3 ttlSecondsAfterFinished: 60 template: metadata: creationTimestamp: null spec: containers: - command: - sleep - \u0026#34;5\u0026#34; image: busybox name: paralleljob resources: {} restartPolicy: Never status: {} After generating the YAML file we\u0026rsquo;ve added the completions, parallelism and ttlSecondsAfterFinished values.\nUntil the Job has completed 6 times, the Job will make sure that 3 Pods are running the Job at all times. When one Pod finished a new Pod is started. At completion of the Job, 6 Pods will have been created by the Job. The Job and Pods are deleted after 60 seconds.\nstudent@minikube:~$ kubectl create -f paralleljob.yml job.batch/paralleljob created student@minikube:~$ kubectl get jobs,pods NAME COMPLETIONS DURATION AGE job.batch/paralleljob 6/6 29s 29s NAME READY STATUS RESTARTS AGE pod/paralleljob-6s9l4 0/1 Completed 0 11s pod/paralleljob-7swk6 0/1 Completed 0 19s pod/paralleljob-8pzgp 0/1 Completed 0 14s pod/paralleljob-ldmtf 0/1 Completed 0 29s pod/paralleljob-tsk4q 0/1 Completed 0 29s pod/paralleljob-x6p8k 0/1 Completed 0 29s student@minikube:~$ kubectl get jobs,pods No resources found in default namespace. Managing Cronjobs # While Jobs are used to run a task a specific number of times, CronJobs are used for tasks that are recurrent or that need to run on a regular basis. In that sense they are very similar to Linux cronjobs.\nWhen running a CronJob, a Job is scheduled and in turn the Job will start a Pod.\nLet\u0026rsquo;s go over this in detail:\nstudent@minikube:~$ kubectl create cronjob -h | less # Create a cron job with a command kubectl create cronjob my-job --image=busybox --schedule=\u0026#34;*/1 * * * *\u0026#34; -- date ... student@minikube:~$ kubectl create cronjob runme --image=busybox --schedule=\u0026#34;*/1 * * * *\u0026#34; -- echo Hello there! cronjob.batch/runme created student@minikube:~$ kubectl get cronjobs,jobs,pods NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE cronjob.batch/runme */1 * * * * False 0 \u0026lt;none\u0026gt; 15s student@minikube:~$ kubectl get cronjobs,jobs,pods NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE cronjob.batch/runme */1 * * * * False 1 8s 28s NAME COMPLETIONS DURATION AGE job.batch/runme-27480120 0/1 8s 8s NAME READY STATUS RESTARTS AGE pod/runme-27480120-xv5l4 0/1 ContainerCreating 0 8s student@minikube:~$ kubectl get cronjobs,jobs,pods NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE cronjob.batch/runme */1 * * * * False 1 2s 82s NAME COMPLETIONS DURATION AGE job.batch/runme-27480120 1/1 13s 62s job.batch/runme-27480121 0/1 2s 2s NAME READY STATUS RESTARTS AGE pod/runme-27480120-xv5l4 0/1 Completed 0 62s pod/runme-27480121-nljcn 0/1 ContainerCreating 0 2s As you can see above, the cronjob.batch/runme cronjob will create a new Job at the top of each minute and each Job will create a new Pod which will run until completion of the task.\nstudent@minikube:~$ kubectl delete cronjob runme cronjob.batch \u0026#34;runme\u0026#34; deleted student@minikube:~$ kubectl get cronjobs,jobs,pods No resources found in default namespace. Resource Requests and Limits # By default, a Pod will consume as much CPU and memory as necessary.\nWe can however use pod.spec.containers.resources to limit usage of those on a per container basis. CPU and memory limitations are the most common, but there are others.\nEach container can has its CPU and memory usage restricted by:\nRequest: kube-scheduler will look for a worker-node that has this amount of resources available and schedule the Pod to run there. It\u0026rsquo;s allowed for a container to use more resources than defined here. If no suitable worker-node is found, the Pod status remains in Pending. Limit: This is a hard limit. If configured, the container runtime prevents the container from using more than the configured resource limit. For the memory resource type, this could result in an out of memory error if the container attempts to consume more memory than allowed. a Pod resource request/limit is the sum of the resource requests/limits for each resource type and for each container in the Pod.\nCPU limits are expressed in millicore or millicpu: 1/1000 of a CPU core.\nSo 500m is 0.5 CPU and 2000m is 2 CPU.\nExample:\n--- apiVersion: v1 kind: Pod metadata: name: frontend spec: containers: - name: db image: mariadb env: - name: MYSQL_ROOT_PASSWORD value: \u0026#34;password\u0026#34; resources: requests: memory: \u0026#34;64Mi\u0026#34; cpu: \u0026#34;250m\u0026#34; limits: memory: \u0026#34;128Mi\u0026#34; cpu: \u0026#34;500m\u0026#34; - name: wordpress image: wordpress resources: requests: memory: \u0026#34;64Mi\u0026#34; cpu: \u0026#34;250m\u0026#34; limits: memory: \u0026#34;128Mi\u0026#34; cpu: \u0026#34;500m\u0026#34; ","date":"23 August 2024","externalUrl":null,"permalink":"/kubernetes-101-kubernetes-essentials/","section":"Blog","summary":"","title":"Kubernetes 101: Kubernetes Essentials","type":"posts"},{"content":" What is Kubernetes? # https://kubernetes.io/\nKubernetes is an open-source ecosystem for automating deployment, scaling and managing of containerized applications. It provides a core solution with many third-party add-ons focusing on different areas:\nNetworking Ingress Monitoring Packaging \u0026hellip; Kubernetes has its origins at Google where it was known as Borg. It\u0026rsquo;s currently owned by the Cloud Native Computing Foundation, an open-source foundation within the Linux Foundation.\nVanilla Kubernetes is Kubernetes directly created from the source code hosted by the CNCF. Different Kubernetes distributions exist that add specific functionality and a selection of solutions from the ecosystem:\nGoogle Anthos Red Hat OpenShift Suse Rancher Canonical Kubernetes \u0026hellip; A new release of Kubernetes is published every 3 months. When a new release is published, new versions of the API (more on that later) may become available and old features may get deprecated. If a feature is deprecated it\u0026rsquo;s important to adopt the new method: because of the 3 month release cycle, the feature will go away within the next 2 releases.\nKubernetes Architecture # Kubernetes has the following main components:\nControl Plane and worker nodes Operators (aka \u0026ldquo;control loop\u0026rdquo;, \u0026ldquo;watch-loops\u0026rdquo; or \u0026ldquo;controller\u0026rdquo;) Services Pods of containers Namespaces and quotas Network and policies Storage. A Kubernetes cluster is made of a Control Plane node and a set of worker nodes. The cluster is driven via API calls to operators.\nThe Control Plane Node # The various components responsible for ensuring that the current state of the cluster matches the desired state are called the Control Plane.\nkube-apiserver # The kube-apiserver is central to the operation of the Kubernetes cluster and exposes the Kubernetes API. You can communicate with the API using a local client called kubectl or you can write your own client and use curl commands.All actions are accepted and validated by this component, and it is the only connection to the etcd database.\nkube-scheduler # The kube-scheduler determines which node will host a Pod of containers. The scheduler will try to view available resources and then try to deploy the Pod based on availability and success.\netcd database # The state of the cluster, networking, and other persistent information is kept in an etcd database. etcd is a strongly consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. This database is only accessible by kube-apiserver.\nkube-controller-manager # Orchestration is managed through a series of watch-loops or control loops, also called controllers or operators. A control loop is a non-terminating loop that regulates the state of a system. Each controller interrogates the kube-apiserver for a particular object state, then modifies the object until the declared state matches the current state. These controllers are compiled into the kube-controller-manager, but others can be added using custom resource definitions.\nThe kube-controller-manager is a core control loop daemon which interacts with the kube-apiserver to determine the state of the cluster. If the state does not match, the manager will contact the necessary controller to match the desired state.\nWorker Nodes # A Worker Node consists of components that maintain running pods.\nkubelet # The kubelet systemd process interacts with the underlying container engine. It accepts the API calls for Pod specifications and it will configure the local node until the specification has been met by passing requests to the local container engine.\nkube-proxy # The kube-proxy creates and manages networking rules to expose the container on the network to other containers or the outside world.\nContainer runtime # The container runtime or container engine is responsible for running containers.\nEach Worker Node could run a different engine if needed: Docker, containerd, CRI-O, podman, \u0026hellip;\nThe Most Essential API Resources # Deployment # The default operator for containers is a Deployment. A Deployment does not directly work with pods, instead it manages ReplicaSets. The ReplicaSet is an operator which will create or terminate pods according to a podSpec. The podSpec is sent to the kubelet, which then interacts with the container engine to download and make the required resources available, then spawn or terminate containers until the status matches the spec.\nPod # Containers are not managed individually, instead, they are part of a larger object called a Pod. A Pod consists of one or more containers which share an IP address, access to storage and namespace. Typically, one container in a Pod runs an application, while other containers support the primary application.\nService # The service operator requests existing IP addresses and information from the endpoint operator, and will manage the network connectivity based on labels. A service is used to communicate between pods, namespaces, and outside the cluster.\nCreating a Lab Environment # The Kubernetes 101 series of articles that I will be publishing over time are meant to provide a basic introduction to Kubernetes. As such, we\u0026rsquo;ll not be using a full blown Kubernetes cluster but we\u0026rsquo;ll be relying on Minikube instead.\nWith Minikube we can quickly and easily setup a local Kubernetes cluster and focus on learning the basics. In a later series, we\u0026rsquo;ll deep dive into a full blown Kubernetes cluster with multiple worker nodes.\nWe will be installing Minikube in an Ubuntu virtual machine with 4GiB of RAM and 2vCPUs and we\u0026rsquo;ll be using Docker as the container engine, so make sure you install Docker as well. Once your virtual machine is ready, head over to the Minikube installation instructions. Make sure you start a cluster, install kubectl and create an alias for it to make life easier.\nVerifying Minikube is working # The minikube command has different options, here\u0026rsquo;s an overview of the commonly used ones:\nminikube status: Gets the status of a local Kubernetes cluster. minikube start: Starts a local Kubernetes cluster. minikube stop: Stops a running local Kubernetes cluster. minikube ssh: Log into the minikube environment (for debugging) minikube dashboard: Opens the Kubernetes dashboard in the local browser. minikube delete: Deletes a local Kubernetes cluster. minikube ip: Retrieves the IP address of the specified node. minikube version: Print the version of minikube. You can see all available options by using the minikube --help command.\nThese will come in handy as well:\nkubectl get all: Display all resources. docker ps: List containers. Bash Completion # Bash completion for kubectl will come in handy. The kubectl completion -h command has instructions for different shells like zsh and fish. Below are the instructions for bash:\napt install bash-completion -y echo \u0026#34;source \u0026lt;(kubectl completion bash)\u0026#34; \u0026gt;\u0026gt; ~/.bashrc source ~/.bashrc Running an application # Let\u0026rsquo;s go over the steps of starting our cluster and launching a simple Nginx Pod:\n# We start our Minikube cluster student@minikube:~$ minikube start ... # Install kubectl student@minikube:~$ minikube kubectl -- get pods -A ... # Verify the status student@minikube:~$ minikube status minikube type: Control Plane host: Running kubelet: Running apiserver: Running kubeconfig: Configured # List all Docker containers - See how Minikube is running a Kubernetes cluster inside a single Docker container student@minikube:~$ docker ps ... # Have a look at the different Kubernetes components which are running in Pods inside the kube-system namespace. student@minikube:~$ kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-64897985d-sj5lw 1/1 Running 0 11m etcd-minikube 1/1 Running 0 11m kube-apiserver-minikube 1/1 Running 0 11m kube-controller-manager-minikube 1/1 Running 0 11m kube-proxy-mgcrk 1/1 Running 0 11m kube-scheduler-minikube 1/1 Running 0 11m storage-provisioner 1/1 Running 1 (10m ago) 11m # Let\u0026#39;s run an Nginx Pod student@minikube:~$ kubectl run nginx --image=nginx pod/nginx created student@minikube:~$ kubectl get pods NAME READY STATUS RESTARTS AGE nginx 1/1 Running 0 24s student@minikube:~$ kubectl get all NAME READY STATUS RESTARTS AGE pod/nginx 1/1 Running 0 37s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 13m Play around with the different minikube commands, and, once done head over to the next article.\n","date":"12 June 2024","externalUrl":null,"permalink":"/kubernetes-101-understanding-kubernetes/","section":"Blog","summary":"","title":"Kubernetes 101: Understanding Kubernetes","type":"posts"},{"content":"","date":"22 March 2023","externalUrl":null,"permalink":"/tags/lxd/","section":"Tags","summary":"","title":"LXD","type":"tags"},{"content":"","date":"22 March 2023","externalUrl":null,"permalink":"/tags/mariadb/","section":"Tags","summary":"","title":"MariaDB","type":"tags"},{"content":"","date":"22 March 2023","externalUrl":null,"permalink":"/tags/php-fpm/","section":"Tags","summary":"","title":"PHP-FPM","type":"tags"},{"content":"","date":"22 March 2023","externalUrl":null,"permalink":"/tags/podman/","section":"Tags","summary":"","title":"Podman","type":"tags"},{"content":" Typically an application container runs a single service, but instead of breaking apart existing multi-serivce applications into microservices (and connecting them with e.g. Kubernetes or OpenShift), we can use Podman (in contrast to Docker) to run multi-service containers using Systemd. Basically we would achieve something similar to LXD system containers but with Podman.\nPodman understands what Systemd needs to do to run in a container. When Podman starts a container that is running init or systemd as its initial command, Podman automatically sets up the tmpfs and cgroups so that Systemd can start succesfully.\nSystemd attempts to write to the cgroup file system. By default, containers cannot write to the cgroup file system when SELinux is enabled. The container_manage_cgroup boolean must be enabled for this to be allowed on a SELinux enforced system: setsebool -P container_manage_cgroup true\nIn this post I\u0026rsquo;ll create a rather basic multi-service container based on the Fedora container image which will be running Nginx, MariaDB and PHP-FPM to serve up a WordPress site with persistent storage both for the document root and the database.\nI\u0026rsquo;ve pushed the final version of the image I\u0026rsquo;ve build below to my Quay.io repository.\nStep 1 - Test Nginx # [student@server1 ~]$ cat Dockerfile FROM fedora MAINTAINER Joeri Smissaert RUN dnf -y upgrade; dnf -y install nginx; dnf clean all; systemctl enable nginx RUN mkdir -p /var/www/worpdress.server1.local/public RUN mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf.backup ADD https://gist.githubusercontent.com/smissaertj/9d02fd974b64fd1a30fd905bc730a098/raw/dee50eb0bea7b93acb6ad0ddb6894cefb74c9d45/nginx.conf /etc/nginx/nginx.conf EXPOSE 80 CMD [\u0026#34;/sbin/init\u0026#34;] Let\u0026rsquo;s build the image:\npodman build -t fedora_wordpress . \u0026hellip;start the container:\n[student@server1 ~]$ podman run -d --name test -p 8080:80 -v /home/student/html:/var/www/wordpress.server1.local/public:Z fedora_wordpress ... \u0026hellip;create a test file in the bind mounted document root and test using cURL:\n[student@server1 ~]$ mkdir html [student@server1 ~]$ echo \u0026#34;JOERI\u0026#34; \u0026gt; html/index.html [student@server1 ~]$ curl localhost:8080 JOERI [student@server1 ~]$ echo \u0026#34;TEST 123\u0026#34; \u0026gt; html/index.html [student@server1 ~]$ curl localhost:8080 TEST 123 So far so good :)\nStep 2 - Test PHP-FPM # In this step we only install and enable PHP-FPM. If the test fails, then I need to revise my Nginx and/or PHP-FPM pool configuration. My Nginx configuration file is custom, while I left the default PHP-FPM configuration file in place.\n[student@server1 ~]$ cat Dockerfile FROM fedora MAINTAINER Joeri Smissaert RUN dnf -y upgrade; dnf -y install nginx php-fpm php-mysqlnd php-pdo php-json; dnf clean all; systemctl enable nginx; systemctl enable php-fpm RUN mkdir -p /var/www/worpdress.server1.local/public RUN mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf.backup ADD https://gist.githubusercontent.com/smissaertj/9d02fd974b64fd1a30fd905bc730a098/raw/dee50eb0bea7b93acb6ad0ddb6894cefb74c9d45/nginx.conf /etc/nginx/nginx.conf EXPOSE 80 CMD [\u0026#34;/sbin/init\u0026#34;] Adjust original Dockerfile with the modifications above, then rebuild the image and run the container:\n[student@server1 ~]$ podman build -t fedora_wordpress . ... [student@server1 ~]$ podman run -d --name test -p 8080:80 -v /home/student/html:/var/www/wordpress.server1.local/public:Z fedora_wordpress ... Remove the html/index.html file and create an html/index.php file with the following content:\n[student@server1 ~]$ cat html/index.php \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;PHP Test\u0026lt;/title\u0026gt; \u0026lt;/head\u0026gt; \u0026lt;body\u0026gt; \u0026lt;?php echo \u0026#39;\u0026lt;p\u0026gt;Hello World\u0026lt;/p\u0026gt;\u0026#39;; ?\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; When we run a cURL test, we should not be seeing the \u0026lt;?php and ?\u0026gt; tags, indicating that our php code was succesfully parsed by PHP-FPM:\n[student@server1 ~]$ curl localhost:8080 \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;PHP Test\u0026lt;/title\u0026gt; \u0026lt;/head\u0026gt; \u0026lt;body\u0026gt; \u0026lt;p\u0026gt;Hello World\u0026lt;/p\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; Yaay! :D\nStep 3 - Test MariaDB # I\u0026rsquo;ll create persistent storage for the database by means of a podman volume:\n[student@server1 ~]$ podman volume create wordpress_db wordpress_db [student@server1 ~]$ podman volume ls DRIVER VOLUME NAME local wordpress_db Again, we adjust our Dockerfile and rebuild our custom image:\nFROM fedora MAINTAINER Joeri Smissaert RUN dnf -y upgrade; dnf -y install nginx php-fpm php-fpm php-mysqlnd php-pdo php-json mariadb-server; dnf clean all; systemctl enable nginx; systemctl enable php-fpm; systemctl enable mariadb RUN mkdir -p /var/www/worpdress.server1.local/public RUN mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf.backup ADD https://gist.githubusercontent.com/smissaertj/9d02fd974b64fd1a30fd905bc730a098/raw/dee50eb0bea7b93acb6ad0ddb6894cefb74c9d45/nginx.conf /etc/nginx/nginx.conf EXPOSE 80 CMD [\u0026#34;/sbin/init\u0026#34;] We run the container:\n[student@server1 ~]$ podman run -d --name test -v wordpress_db:/var/lib/mysql:Z fedora_wordpress ... Next, we create the database and configure the database user and password:\n[student@server1 ~]$ podman exec test mysql -e \u0026#34;create database wordpressdb;\u0026#34; [student@server1 ~]$ podman exec test mysql -e \u0026#34;grant all privileges on wordpressdb.* to \u0026#39;wordpress\u0026#39;@\u0026#39;localhost\u0026#39; identified by \u0026#39;password\u0026#39;;\u0026#34; We can now move on to the next step and install WordPress.\nStep 4 - Install WordPress # In this step I\u0026rsquo;ll move away from a bind mounted directory (used during Step 1 and Step 2) to a podman volume to persistently store the WordPress files.\n[student@server1 ~]$ podman volume create wordpress_files wordpress_files [student@server1 ~]$ podman volume ls DRIVER VOLUME NAME local wordpress_files local wordpress_db [student@server1 ~]$ podman volume inspect wordpress_files [ { \u0026#34;Name\u0026#34;: \u0026#34;wordpress_files\u0026#34;, \u0026#34;Driver\u0026#34;: \u0026#34;local\u0026#34;, \u0026#34;Mountpoint\u0026#34;: \u0026#34;/home/student/.local/share/containers/storage/volumes/wordpress_files/_data\u0026#34;, \u0026#34;CreatedAt\u0026#34;: \u0026#34;2021-04-14T00:25:06.906021271+04:00\u0026#34;, \u0026#34;Labels\u0026#34;: { }, \u0026#34;Scope\u0026#34;: \u0026#34;local\u0026#34;, \u0026#34;Options\u0026#34;: { }, \u0026#34;UID\u0026#34;: 0, \u0026#34;GID\u0026#34;: 0, \u0026#34;Anonymous\u0026#34;: false } ] From the last command we can see where exactly the data will be stored:\nMountpoint\u0026#34;: \u0026#34;/home/student/.local/share/containers/storage/volumes/wordpress_files/_data\u0026#34; I\u0026rsquo;ll go ahead and extract WordPress inside that directory:\n[student@server1 ~]$ cd ~/.local/share/containers/storage/volumes/wordpress_files/_data/ [student@server1 _data]$ wget https://wordpress.org/latest.tar.gz ... [student@server1 _data]$ tar xf latest.tar.gz --strip-components 1 [student@server1 _data]$ rm -rf latest.tar.gz Let\u0026rsquo;s start the container and test our installation:\n[student@server1 ~]$ podman run -d --name wordpress_test_container -p 8080:80 -v wordpress_db:/var/lib/mysql:Z -v wordpress_files:/var/www/wordpress.server1.local/public:Z fedora_wordpress bc006160ad6b74b81fa3fc353bc0cbb1cec3b394365dc98259984a86c971cd9f [student@server1 ~]$ Testing with cURL seems to go fine:\n[student@server1 ~]$ curl -I localhost:8080 HTTP/1.1 302 Found Server: nginx/1.18.0 Date: Tue, 13 Apr 2021 20:34:46 GMT Content-Type: text/html; charset=UTF-8 Connection: keep-alive X-Powered-By: PHP/7.4.16 Location: http://localhost:8080/wp-admin/setup-config.php So at this point we have a working WordPress multi-service container :D\nBelow is the final version of our Dockerfile: # FROM fedora MAINTAINER Joeri Smissaert RUN dnf -y upgrade; dnf -y install nginx php-fpm php-fpm php-mysqlnd php-pdo php-json mariadb-server; dnf clean all; systemctl enable nginx; systemctl enable php-fpm; systemctl enable mariadb RUN mkdir -p /var/www/worpdress.server1.local/public RUN mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf.backup ADD https://gist.githubusercontent.com/smissaertj/9d02fd974b64fd1a30fd905bc730a098/raw/dee50eb0bea7b93acb6ad0ddb6894cefb74c9d45/nginx.conf /etc/nginx/nginx.conf EXPOSE 80 CMD [\u0026#34;/sbin/init\u0026#34;] I\u0026rsquo;ve pushed the final version of the image I\u0026rsquo;ve build to my Quay.io repository.\nAs long as we keep the wordpress_files and wordpress_db volumes, I can destroy the running container and recreate it without any effect on the data:\npodman run -d --name container_name -p 8080:80 -v wordpress_files:/var/www/wordpress.server1.local/public:Z -v wordpress_db:/var/lib/mysql:Z quay.io/smissaertj/fedora_wordpress Finally, I want my WordPress site to start at boot even when I\u0026rsquo;m not logging in to my machine as the user which created the container:\n[student@server1 ~]$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d4fb97ba659d localhost/fedora_wordpress:latest /sbin/init 6 minutes ago Up 6 minutes ago 0.0.0.0:8080-\u0026gt;80/tcp wordpress_test [student@server1 ~]$ mkdir -p ~/.config/systemd/user [student@server1 ~]$ cd .config/systemd/user/ [student@server1 user]$ podman generate systemd --name wordpress_test --files --new /home/student/.config/systemd/user/container-wordpress_test.service [student@server1 user]$ su - root Password: [root@server1 ~]# loginctl enable-linger student [root@server1 ~]# exit logout [student@server1 user]$ systemctl --user daemon-reload [student@server1 user]$ systemctl --user enable container-wordpress_test.service Created symlink /home/student/.config/systemd/user/multi-user.target.wants/container-wordpress_test.service → /home/student/.config/systemd/user/container-wordpress_test.service. Created symlink /home/student/.config/systemd/user/default.target.wants/container-wordpress_test.service → /home/student/.config/systemd/user/container-wordpress_test.service. [student@server1 user]$ reboot The --new option passed to the podman generate systemd command will make sure that the container is destroyed when the service stops and recreated when the service starts.\n","date":"22 March 2023","externalUrl":null,"permalink":"/podman-102-wordpress-multi-service-container/","section":"Blog","summary":"","title":"Podman 102: Building a WordPress multi-service container with Nginx, PHP-FPM and MariaDB","type":"posts"},{"content":"","date":"22 March 2023","externalUrl":null,"permalink":"/tags/rhel/","section":"Tags","summary":"","title":"RHEL","type":"tags"},{"content":"","date":"22 March 2023","externalUrl":null,"permalink":"/tags/systemd/","section":"Tags","summary":"","title":"Systemd","type":"tags"},{"content":"","date":"22 March 2023","externalUrl":null,"permalink":"/tags/wordpress/","section":"Tags","summary":"","title":"WordPress","type":"tags"},{"content":"","date":"21 December 2022","externalUrl":null,"permalink":"/tags/buildah/","section":"Tags","summary":"","title":"Buildah","type":"tags"},{"content":"","date":"21 December 2022","externalUrl":null,"permalink":"/tags/oci/","section":"Tags","summary":"","title":"OCI","type":"tags"},{"content":" Understanding Containers # For a data center to operate efficiently, its machines and running components on those machines must become as generic and as much automated as possible. We can partly achieve this by seperating the applications from the operating system. This means not just packaging applications into things we install (like RPM or Deb packages), but also putting together sets of software into packages that themselves can run in ways that keep them independent and seperate from the operating system. Virtual Machines and Containers are two ways of packaging sets of software and their dependencies in a way which is separated from the host operating system they are running on.\nA virtual machine is a complete operating system that runs on another operating sytem, you can have many virtual machines on one physical computer. Everything an application or service needs to run can be stored inside that virtual machine or in attached storage. A virtual machine has its own kernel, file system, process table, network interfaces and other operating system features separate from the host, while sharing CPU and RAM with the host system. A VM sees an emulation of the computer hardware and not the host hardware directly, hence the term virtual machine.\nA Container is similar to a virtual machine, except that it doesn\u0026rsquo;t have its own kernel. It remains separate from the host system by using its own set of namespaces. Just like a VM, you can move it from one host to another to run it wherever it is convenient. Typically you would build your own container images by getting a secure base image and then adding your own layers of software on top of that image to create a new image. To share your image, you push them to shared container registries from where others are allowed to pull them.\nContainers run on top of a container engine, like Docker, CRI-O (which is the default on RHEL 8), Moby or rkt, and typically a container runs a single application or service (which can be connected in microservices using OpenShift or Kubernetes for example), although there are systemd images from which you can build multiservice containers.\nPodman is a daemonless container engine that is compatible with Docker, for developing, managing, and running Open Container Initiative (OCI) containers and container images on Linux.\nNamespaces # Linux support for namespaces is what allows containers to be contained. With namespaces, the Linux kernel can associate one or more processes with a set of resources. Normal processes, not run in a container, use the same host namespaces. By default, processes in a container can only see the container\u0026rsquo;s namespaces and not those of the host.\nProcess table - A container has its own set of process IDs and, by default, can only see processes running inside the container. While PID 1 on the host is the init (systemd) process, in a container PID 1 is the first process run inside the container.\nNetwork interfaces - By default, a container has a single network interface and is assigned an IP address when the container runs. A service run inside a container is not exposed outside of the host system, by default. You can have hundreds of webservers running on the same host without conflict, but you need to manage how those ports are exposed outside of the host.\nMount table - By default, a container can\u0026rsquo;t see the host\u0026rsquo;s root file system or any other mounted file system listed in the host\u0026rsquo;s mount table. Files or directories needed from the host can be selectively bind-mounted inside the container.\nUser IDs - Containerized processes run as some UID within the host\u0026rsquo;s namespace, and, with another set of UIDs nested within the container. This can, for example, let a process run as root within the container but not have any special privileges to the host system.\nUTS - The UNIX Time Sharing namespace allows a containerized process to have a different host and domain name from the host.\nControl Group - A containerized process runs within a selected cgroup and cannot see the other cgroups available on the host system. Similarly, it cannot see the identify of its own cgroup. Control Groups are used for resource management.\nInterprocess Communications - A containerized process cannot see the IPC namespace of the host.\nAlthough access to any host namespace is restricted by default, privileges to host namespaces can be opened selectively. In that way, you can do things like mount configuration files or data inside the container and map container ports to host ports to expose services outside of the host.\nContainer Registries # Permanent storage for containers is done in what is referred to as a container registry. When you create a container image that you want to share, you can push that image to a public or private (which you maintain yourself) container registry. Someone who wants to use your container image will then pull it from the registry.\nLarge public container image registries are, for example, Docker hub and Quay Registry.\nBase Images and Layers # Although you can create containers from scratch, most often a container is built by starting with a well-known base image and adding software to it. Linux distributions offer base images in different forms, like standard and minimal versions. But there are also base images you can build on that offer runtimes for PHP, Java and other development environments.\nRed Hat offers freely available Universal Base Images (UBIs) for standard, minimal and a variety of runtime containers. You can find those by searching the Red Hat Container Catalog.\nYou can add software to a base image by defining the build using yum commands to install software from software repositories into the new container. When you add software to an image, it creates a new layer that becomes part of the new image. You can reuse the same base image for all container you build, only one copy of the base image is needed on the host. If you\u0026rsquo;re running 10 different containers based on the same base image, you only need to pull and store the base image once. For each new image you build, you only add the data that differs from the base image.\nRunning and Managing Containers with Podman # Pulling and Running Containers # In order to start using containers with podman, we need to install the container-tools module:\n[root@server1 student]# yum module install container-tools ... Let\u0026rsquo;s choose a reliable image to try out, one that comes from an official project, is up to date and has been scanned for vulnerabilities:\n[student@server1 ~]$ podman pull registry.access.redhat.com/ubi8/ubi Trying to pull registry.access.redhat.com/ubi8/ubi... Getting image source signatures Copying blob 64607cc74f9c done Copying blob 13897c84ca57 done Copying config 9992f11c61 done Writing manifest to image destination Storing signatures 9992f11c61c5fa38a691f80c7e13b75960b536aade4cce8543433b24623bce68 [student@server1 ~]$ We can verify that the image is on our system using the podman images command:\n[student@server1 ~]$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE registry.access.redhat.com/ubi8/ubi latest 9992f11c61c5 11 days ago 213 MB Next, let\u0026rsquo;s start an interactive shell from this base image. We use the podman run command, specify the -i (interactive) and -t (terminal) options, followed by the name of the image (ubi) and the command we wish to start once the container is up and running (bash):\n[student@server1 ~]$ podman run -it ubi bash [root@888b3cbea5cc /]# We are in an interactive session within the container from the bash shell. Notice the container is using the host kernel:\n[root@888b3cbea5cc /]# ls bin boot dev etc home lib lib64 lost+found media mnt opt proc root run sbin srv sys tmp usr var [root@888b3cbea5cc /]# cat /etc/os-release | grep -i ^NAME NAME=\u0026#34;Red Hat Enterprise Linux\u0026#34; [root@888b3cbea5cc /]# uname -r 4.18.0-240.el8.x86_64 We can add software to the container:\n[root@888b3cbea5cc /]# yum install procps -y ... [root@888b3cbea5cc /]# ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 13:13 pts/0 00:00:00 bash root 39 1 0 13:20 pts/0 00:00:00 ps -ef Notice that form within the container, we only see two running processes: the shell and the ps command. PID 1 is the bash shell.\nWe can exit the container by using the exit command. The container is now no longer running, but it\u0026rsquo;s still available on the host in a stopped state. The podman ps \u0026ndash;all command shows all available containers:\n[student@server1 ~]$ podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES bold_aryabhata 888b3cbea5cc registry.access.redhat.com/ubi8/ubi:latest bash 9 minutes ago Exited (0) 3 seconds ago musing_almeida Managing Container State # Unless you specifically set a container to be removed when it\u0026rsquo;s stopped (--rm option), paused or fails, the container is still on your system. You can see the status of all containers on the system, running or stopped, using the podman ps command:\n[student@server1 ~]$ podman run -d nginx e968c7e569cbe60d909b2108ba5a2067bb3e771327f4729b85566280efe944a6 [student@server1 ~]$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e968c7e569cb docker.io/library/nginx:latest nginx -g daemon o... 4 seconds ago Up 3 seconds ago loving_swartz [student@server1 ~]$ podman stop e968 e968c7e569cbe60d909b2108ba5a2067bb3e771327f4729b85566280efe944a6 [student@server1 ~]$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [student@server1 ~]$ podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e968c7e569cb docker.io/library/nginx:latest nginx -g daemon o... 27 seconds ago Exited (0) 5 seconds ago loving_swartz The podman stop command sends a SIGTERM signal and if the container doesn\u0026rsquo;t stop after 10 seconds it will send a SIGKILL signal. You can also send the SIGKILL signal immediately using the podman kill command. Just like the podman stop command stops a container, you can start a container using podman start or simply restart a container using podman restart.\nLastly, we can delete the container permanently by using the podman rm command:\n[student@server1 ~]$ podman rm e968 e968c7e569cbe60d909b2108ba5a2067bb3e771327f4729b85566280efe944a6 [student@server1 ~]$ podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [student@server1 ~]$ Note that the podman rm command only deletes the container and not the image.\nRunning commands in a container # When we are detached from a container we can still execute commands inside the container using podman exec:\n[student@server1 ~]$ podman exec cd87 cat /etc/os-release | grep ^NAME NAME=\u0026#34;Debian GNU/Linux\u0026#34; Or, we can attach to the container:\n[student@server1 ~]$ podman exec -it cd87 /bin/bash root@cd87164b978f:/# \u0026hellip;and detach using the CTRL-P+Q sequence.\nManaging Container Ports # We can map a host port to the container application port to make the application in the container reachable from the host machine:\n[student@server1 ~]$ podman run -d -p 8000:80 nginx 965fe32d0b4b96d469ddb5638edaa5ac18fe41fc083082844bc8ddae0f6a9a33 [student@server1 ~]$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 965fe32d0b4b docker.io/library/nginx:latest nginx -g daemon o... 3 seconds ago Up 2 seconds ago 0.0.0.0:8000-\u0026gt;80/tcp musing_mclaren [student@server1 ~]$ podman port -a 965fe32d0b4b 80/tcp -\u0026gt; 0.0.0.0:8000 [student@server1 ~]$ podman port 965 80/tcp -\u0026gt; 0.0.0.0:8000 In the example above, we mapped the host port 8000 to port 80 of the container. Note that you can only map container ports to non privileged (\u0026gt;1024) ports on the host when running rootless containers.\nWith the above done, we can curl the host port and see Nginx serving its default content:\n[student@server1 ~]$ curl localhost:8000 \u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;Welcome to nginx!\u0026lt;/title\u0026gt; \u0026lt;style\u0026gt; body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } \u0026lt;/style\u0026gt; \u0026lt;/head\u0026gt; \u0026lt;body\u0026gt; \u0026lt;h1\u0026gt;Welcome to nginx!\u0026lt;/h1\u0026gt; \u0026lt;p\u0026gt;If you see this page, the nginx web server is successfully installed and working. Further configuration is required.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;For online documentation and support please refer to \u0026lt;a href=\u0026#34;http://nginx.org/\u0026#34;\u0026gt;nginx.org\u0026lt;/a\u0026gt;.\u0026lt;br/\u0026gt; Commercial support is available at \u0026lt;a href=\u0026#34;http://nginx.com/\u0026#34;\u0026gt;nginx.com\u0026lt;/a\u0026gt;.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;\u0026lt;em\u0026gt;Thank you for using nginx.\u0026lt;/em\u0026gt;\u0026lt;/p\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; Now, if we would want access from outside of the host machine, we should not forget to configure the host machine\u0026rsquo;s firewall:\n[student@server1 ~]$ su - root Password: [root@server1 ~]# firewall-cmd --add-port=8000/tcp --permanent \u0026amp;\u0026amp; firewall-cmd --reload success success [root@server1 ~]# exit logout [student@server1 ~]$ By default podman runs rootless containers. Rootless containers cannot bind to a privileged port and do NOT have an IP address, you would need port forwarding instead. If you need a container with an IP address, you need a root container: sudo podman run -d nginx\nAttaching Storage to Containers # Storage in containers is ephemeral: modifications are written to the container writeable layer and stay around for the container lifetime. For persistent storage needs, we use bind mounts to connect a directory inside the container to a directory on the host machine.\nWe start preparing on the hostmachine, creating directories, setting basic permissions and changing the SELinux file context type to container_file_t. SELinux is very important when using root containers, as without, the root container will have access to the entire host file system.\nI\u0026rsquo;ll run through an example where we set the document root of the nginx image to the /home/student/html directory on the host machine. Inside that directory we\u0026rsquo;ll create a basic html file that the nginx container is going to serve.\nPreparing Host Storage # [root@server1 student]# pwd /home/student [student@server1 ~]$ ls -l total 0 drwxrwxr-x. 2 student student 6 Apr 12 21:29 html [root@server1 student]# semanage fcontext -a -t container_file_t \u0026#34;/home/student/html(/.*)?\u0026#34; [root@server1 student]# restorecon -Rv /home/student/html Relabeled /home/student/html from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:container_file_t:s0 Mounting Storage Inside the Container. # At this point we can delete the container from the previous example, start a new container and bind mount the host directory /home/student/html to the default document root of Nginx in the container: /usr/share/nginx/html\nIf the container user is owner of the host directory, the :Z (SELinux) option can be used: podman run -d --name web1 -p 8000:80 -v /home/student/html:/usr/share/nginx/html:Z nginx\n--d we run the container in detached mode. --name we set a name for our new container. -p we map the host port to the container port. -v we bind a host directory to a directory inside the container. nginx the name of the image we use to start our container from. [student@server1 ~]$ podman run -d --name web1 -p 8000:80 -v /home/student/html:/usr/share/nginx/html:Z nginx 1988217288c55050a2820881ccf75e4436097d8128f9d2dec8a08af6674c6f88 [student@server1 ~]$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1988217288c5 docker.io/library/nginx:latest nginx -g daemon o... 4 seconds ago Up 4 seconds ago 0.0.0.0:8000-\u0026gt;80/tcp web1 [student@server1 ~]$ curl localhost:8000 \u0026lt;html\u0026gt; \u0026lt;head\u0026gt;\u0026lt;title\u0026gt;403 Forbidden\u0026lt;/title\u0026gt;\u0026lt;/head\u0026gt; \u0026lt;body\u0026gt; \u0026lt;center\u0026gt;\u0026lt;h1\u0026gt;403 Forbidden\u0026lt;/h1\u0026gt;\u0026lt;/center\u0026gt; \u0026lt;hr\u0026gt;\u0026lt;center\u0026gt;nginx/1.19.9\u0026lt;/center\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; After starting the container, you\u0026rsquo;ll see that the curl test now returns a 403 Forbidden status. This is because the Nginx document root is bound to an empty directory on our host machine. Let\u0026rsquo;s create an html file for Nginx to serve:\n[student@server1 ~]$ echo \u0026#34;\u0026lt;h1\u0026gt;TEST NGINX\u0026lt;/h1\u0026gt;\u0026#34; \u0026gt; html/index.html [student@server1 ~]$ curl localhost:8000 \u0026lt;h1\u0026gt;TEST NGINX\u0026lt;/h1\u0026gt; [student@server1 ~]$ At this point we can manage the content that Nginx is serving directly from the host machine.\nEnvironment Variables # Podman allows us to set arbitrary environment variables that will become available to processes running in the container:\npodman run -d --name mydb -e MYSQL_ROOT_PASSWORD=password -e MYSQL_USER=student -e MYSQL_PASSWORD=password -e MYSQL_DATABASE=studentdb -p 3306:3306 mariadb Using the -e option, in the above example, we set the MySQL root password, user, password and database name. If we don\u0026rsquo;t specify a value for a variable, then podman will look for the value in the host environment and only set it if that variable has a value.\nSimilarly, instead of passing the environment variables one by one, we can define them in a file and then pass the filename to podman using the --env-file option: podman run -d --name mydb --env-file=variables.txt -p 9999:3306 mariadb\n[student@server1 ~]$ cat variables.txt MYSQL_ROOT_PASSWORD=password MYSQL_USER=student MYSQL_PASSWORD=password MYSQL_DATABASE=studentdb We can now connect from the host machine to the MariaDB instance in the container:\n[student@server1 ~]$ podman run -d --name mydb --env-file=variables.txt -p 3306:3306 mariadb bd08dcbd3eef3907423ee2e55164e1e222a511f58a96d2c4e474f4ea8d56235b [student@server1 ~]$ mysql -u student -h 127.0.0.1 -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \\g. Your MySQL connection id is 3 Server version: 5.5.5-10.5.9-MariaDB-1:10.5.9+maria~focal mariadb.org binary distribution Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type \u0026#39;help;\u0026#39; or \u0026#39;\\h\u0026#39; for help. Type \u0026#39;\\c\u0026#39; to clear the current input statement. mysql\u0026gt; show databases; +--------------------+ | Database | +--------------------+ | information_schema | | studentdb | +--------------------+ 2 rows in set (0.00 sec) Some containers require environment variables to run them. If a container fails because of this requirement, use podman logs container_name to see the application log. Alternatively, use podman inspect | grep -i usage.\nManaging Containers as Services # Now that we have a running container, we can auto start it in a stand-alone situation. The container would start running even though the user that is running the container is not logged in. For this we can create systemd user unit files (for rootless containers), and manage them with systemctl.\nSystemd user services start when a user session is opened, and close when the user session is stopped. We need to use the loginctl enable-linger command to start systemd user services at boot without requiring the user to login:\n[root@server1 ~]# loginctl enable-linger student [root@server1 ~]# loginctl show-user student | grep -i ^linger Linger=yes [root@server1 ~]# Next, we use podman generate systemd to generate a user systemd unit file. This will create the file in the working directory. We need to create the ~/.config/systemd/user directory (for a root container what would be in /etc/systemd/system), and move the user unit file into this directory.\n[student@server1 ~]$ mkdir -p ~/.config/systemd/user [student@server1 ~]$ podman generate systemd --name mydb --files /home/student/container-mydb.service [student@server1 ~]$ mv container-mydb.service ~/.config/systemd/user/ [student@server1 ~]$ systemctl --user daemon-reload [student@server1 ~]$ systemctl --user enable container-mydb.service Created symlink /home/student/.config/systemd/user/multi-user.target.wants/container-mydb.service → /home/student/.config/systemd/user/container-mydb.service. Created symlink /home/student/.config/systemd/user/default.target.wants/container-mydb.service → /home/student/.config/systemd/user/container-mydb.service. [student@server1 ~]$ When we reboot our host machine, the mydb container will automatically start even though the student user is not logged in.\nTo have systemd create the container when the service starts, and delete the container when the service stops, add the --new option. Keep in mind you\u0026rsquo;ll lose all changes if you didn\u0026rsquo;t configure persistent storage for the container:\n[student@server1 ~]$ podman generate systemd --name mydb --files --new Working with Images # An image is a read-only but runnable instance of a container that can be used to build new images. They are obtained from registries which are configured in /etc/containers/registries.conf:\n[student@server1 ~]$ grep -ia1 ^registries /etc/containers/registries.conf [registries.search] registries = [\u0026#39;registry.access.redhat.com\u0026#39;, \u0026#39;registry.redhat.io\u0026#39;, \u0026#39;docker.io\u0026#39;] -- [registries.insecure] registries = [] -- [registries.block] registries = [] Under the [registries.search] value we find an array of registries that will be searched for a specific image in the order they appear in. For example, if you do podman pull nginx, podman will look for the nginx image on registry.access.redhat.com, registry.redhat.io, docker.io subsequently until it finds the image.\nRegistries that do not use TLS when using images, or which are using self-signed certificates need to be placed under [registries.insecure].\nYou can block specific registries under [registries.block], or, if you specify a wildcard (\u0026quot;*\u0026quot;) then all registries are blocked except those that were specified under [registries.search].\nYou can also verify what regestries are in used by issueing the podman info command.\nSearching for images # We use the podman search command to search for images on either all configured registries or only on specific registries. The search results can be filtered using different options as well. A few examples below:\n[student@server1 ~]$ podman search docker.io/nginx --limit 1 INDEX NAME DESCRIPTION STARS OFFICIAL AUTOMATED docker.io docker.io/library/nginx Official build of Nginx. 14707 [OK] [student@server1 ~]$ podman search registry.redhat.io/nginx --limit 1 INDEX NAME DESCRIPTION STARS OFFICIAL AUTOMATED redhat.io registry.redhat.io/rhel8/nginx-116 Platform for running nginx 1.16 or building ... 0 [student@server1 ~]$ podman search docker.io/mariadb --filter is-official=true INDEX NAME DESCRIPTION STARS OFFICIAL AUTOMATED docker.io docker.io/library/mariadb MariaDB Server is a high performing open sou... 4043 [OK] Inspecting Images # Now that we have an idea of what nginx images are available to us, we can inspect them remotely (without pulling them) using skopeo:\n[student@server1 ~]$ skopeo inspect docker://docker.io/nginx { \u0026#34;Name\u0026#34;: \u0026#34;docker.io/library/nginx\u0026#34;, \u0026#34;Digest\u0026#34;: \u0026#34;sha256:6b5f5eec0ac03442f3b186d552ce895dce2a54be6cb834358040404a242fd476\u0026#34;, \u0026#34;RepoTags\u0026#34;: [ \u0026#34;1-alpine-perl\u0026#34;, \u0026#34;1-alpine\u0026#34;, ... Note that the skopeo inspect command always takes the docker:// prefix regardless of what registry the image you\u0026rsquo;re inspecting is located on:\n[student@server1 ~]$ skopeo inspect docker://registry.redhat.io/rhel8/mariadb-103 { \u0026#34;Name\u0026#34;: \u0026#34;registry.redhat.io/rhel8/mariadb-103\u0026#34;, \u0026#34;Digest\u0026#34;: \u0026#34;sha256:c6f117263e36880af79bba1de2018462126d226439d28d074f30bcfaf57dabe1\u0026#34;, \u0026#34;RepoTags\u0026#34;: [ \u0026#34;1-116\u0026#34;, \u0026#34;1-116-source\u0026#34;, ... If we have a local image we wish to inspect, we can use podman inspect instead:\n[student@server1 ~]$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE docker.io/library/nginx latest 519e12e2a84a 3 days ago 137 MB docker.io/library/mariadb latest e76a4b2ed1b4 10 days ago 407 MB registry.access.redhat.com/ubi8/ubi latest 9992f11c61c5 13 days ago 213 MB [student@server1 ~]$ podman inspect registry.access.redhat.com/ubi8/ubi [ { \u0026#34;Id\u0026#34;: \u0026#34;9992f11c61c5fa38a691f80c7e13b75960b536aade4cce8543433b24623bce68\u0026#34;, \u0026#34;Digest\u0026#34;: \u0026#34;sha256:17ff29c0747eade777e8b9868f97ba37e6b8b43f5ed2dbf504ff9277e1c1d1ca\u0026#34;, \u0026#34;RepoTags\u0026#34;: [ \u0026#34;registry.access.redhat.com/ubi8/ubi:latest\u0026#34; ... Removing Images # When new images become available, the old version of the image is kept on your system. We can remove images using the podman rmi command:\n[student@server1 ~]$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE docker.io/library/nginx latest 519e12e2a84a 3 days ago 137 MB docker.io/library/mariadb latest e76a4b2ed1b4 10 days ago 407 MB registry.access.redhat.com/ubi8/ubi latest 9992f11c61c5 13 days ago 213 MB [student@server1 ~]$ podman rmi ubi Untagged: registry.access.redhat.com/ubi8/ubi:latest Deleted: 9992f11c61c5fa38a691f80c7e13b75960b536aade4cce8543433b24623bce68 [student@server1 ~]$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE docker.io/library/nginx latest 519e12e2a84a 3 days ago 137 MB docker.io/library/mariadb latest e76a4b2ed1b4 10 days ago 407 MB Creating Images from a Dockerfile # We can use podman and buildah to create new images from a Dockerfile. The resulting images are OCI compliant, so they will work on any runtime that meets the OCI Runtime Specification (such as Docker and CRI-O).\nIn the below example we prepare a Dockerfile to install the Apache webserver onto a Fedora image and later use podman build to create a new image from this Dockerfile.\n[student@server1 ~]$ cat Dockerfile # Base on the Fedora image FROM fedora:latest MAINTAINER Joeri Smissaert # Update image and install Nginx RUN dnf -y update; dnf -y clean all RUN dnf -y install httpd # Expose the default port 80 EXPOSE 80 # Run Nginx CMD [\u0026#34;/usr/sbin/httpd\u0026#34;,\u0026#34;-DFOREGROUND\u0026#34;] [student@server1 ~]$ podman build -t fedora-apache . ... [student@server1 ~]$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/fedora-apache latest cb083eb46577 15 minutes ago 483 MB [student@server1 ~]$ podman run -d --name myweb1 -p 8080:80 fedora-apache 2f8f1ef6c484f2825f7a11f30c8601799b0736145917f6428b395b4c599cbd6e [student@server1 ~]$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 2f8f1ef6c484 localhost/fedora-apache:latest /usr/sbin/httpd -... 3 seconds ago Up 2 seconds ago 0.0.0.0:8080-\u0026gt;80/tcp myweb1 Tagging and Pushing an Image to a Registry # In this example, I\u0026rsquo;ll tag and push the fedora-apache image to Quay.io.\n[student@server1 ~]$ podman login quay.io Username: ******** Password: Login Succeeded! [student@server1 ~]$ podman tag fedora-apache quay.io/smissaertj/fedora-apache:v1.0 [student@server1 ~]$ podman push quay.io/smissaertj/fedora-apache:v1.0 Getting image source signatures Copying blob 7ddfcddbaf0e done Copying blob dcbc36c2ed7d done Copying blob 6d668c00f3f1 done Copying config cb083eb465 done Writing manifest to image destination Copying config cb083eb465 [--------------------------------------] 0.0b / 1.9KiB Writing manifest to image destination Storing signatures [student@server1 ~]$ You can find the image here: https://quay.io/smissaertj/fedora-apache\n","date":"21 December 2022","externalUrl":null,"permalink":"/podman-101-managing-and-running-containers/","section":"Blog","summary":"","title":"Podman 101: Managing and Running Containers","type":"posts"},{"content":"","date":"21 December 2022","externalUrl":null,"permalink":"/tags/skopeo/","section":"Tags","summary":"","title":"Skopeo","type":"tags"},{"content":" Understanding Local Time # When a Linux machine boots, the hardware clock, also referred to as the real-time clock, is read. This clock resides in the computer hardware, it\u0026rsquo;s in an integrated circuit on the system board that is independent of the current state of the operating system. It keeps running when the computer is shutdown, as long as the system board battery or power supply feeds it. The hardware clock value is known as hardware time, the system gets its initial time setting from hardware time. The hardware clock is usually set to Coordinated Universal Time (UTC).\nSystem time is maintained by the operating system, it\u0026rsquo;s independent of the hardware clock. When the system time is changed, the new system time is not automatically synchronized with the hardware clock.\nSystem time is kept in UTC, applications runing on the operating system convert system time into local time. Local time is the actual time in the current time zone, daylight saving time (DST) is considered so that the system always shows an accurate time.\nConcept Explanation Hardware clock The clock that resides on the main board of a computer system. Real-time clock Same as hardware clock. System time The time that is maintained by the operating system. Software clock Similar to system time. UTC Coordinated Universal Time, a worldwide standard time. Daylight saving time Calculation that is made to change time automatically when DST changes occur. local time The time that corresponds to the time in the current time zone. Using Network Time Protocol # Since the hardware clock is typically part of the computer\u0026rsquo;s motherboard, it can be potentially unreliable. It\u0026rsquo;s a good idea to use time from a more reliable source. Generally speaking, two solutions are available.\nOne option is to buy a more reliable hardware clock. Using an external hardware clock is a common solution in datacenter environments to guarantee reliable time is maintained even if external networks for time synchronization are temporarily not available. An example would be a very accurate atomic clock.\nA more common solution is to configure your machine to use Network Time Protocol (NTP), a method of maintaining system time provided through NTP servers on the Internet. To determine which Internet NTP server should be used, the concept of stratum is introduced. Stratum defines the reliability of an NTP time source, and the lower the stratum value, the more reliable it is. Typically, Internet time servers are using stratum 1 or 2. When you configure a local time server, you can use a higher stratum value. As a consequence, machines configured to use the local time server will only ever use it if Internet time servers (with a lower stratum) are not available.\nSetting up a machine to use NTP on RHEL 8 is easy if the server is already connected to the internet. In this case the /etc/chrony.conf file is prepopulated with a standard list of NTP servers. You would only need to turn on NTP using the timedatectl set-ntp true command (more on this later).\nManaging Time on Red Hat Enterprise Linux # On a Linux system, time is calculated as an offset of epoch time. Epoch time is the number seconds since January 1, 1970, in UTC. You can convert an epoch time stamp to a human readable form using the date --date command, followd by the epoch string starting with an @:\n[student@server1 ~]$ date --date @1420987251 Sun Jan 11 06:40:51 PM +04 2015 Using date # The date command enables you to manage the system time. Or you can use it to show the current time in different formats:\ndate - Shows the current system time. date +%d-%m-%y - Shows the current system day, month and year. date -s 16:03 - Sets the current system time to 3 minutes pas 4pm. Using hwclock # The date command will not change the hardware time. To manage hardware time you can use the hwclock command, which has many options (See hwclock --help). Some options of interest:\nhwclock --systohc - Sync the current system time to the hardware clock. hwclock --hctosys - Sync the current hardware time to the system clock. Using timedatectl # The timedatectl command shows detailed information about the current time and date. It also displays the time zone, in addition to information about the use of NTP network time and DST.\nThe timedatectl command works with the below subcommands to perform time operations:\nCommand Explanation status Shows the current time settings. set-time TIME Sets the current time. set-timezone TIMEZONE Sets the time zone. list-timezone Shows a list of all time zones. set-local-rtc [0 1] set-ntp [0 1] [root@server1 ~]# timedatectl status Local time: Mon 2021-03-15 21:27:17 +04 Universal time: Mon 2021-03-15 17:27:17 UTC RTC time: Mon 2021-03-15 17:27:17 Time zone: Indian/Mauritius (+04, +0400) System clock synchronized: yes NTP service: active RTC in local TZ: no [root@server1 ~]# timedatectl set-time 22:30 [root@server1 ~]# timedatectl Local time: Mon 2021-03-15 22:30:03 +04 Universal time: Mon 2021-03-15 18:30:03 UTC RTC time: Mon 2021-03-15 18:30:03 Time zone: Indian/Mauritius (+04, +0400) System clock synchronized: no NTP service: inactive RTC in local TZ: no After enabling NTP again, you will have to wait a few minutes for the time to synchronize again:\n[root@server1 ~]# timedatectl set-ntp 1 [root@server1 ~]# timedatectl Local time: Mon 2021-03-15 21:30:19 +04 .... [root@server1 ~]# timedatectl list-timezones | grep -i mauritius Indian/Mauritius [root@server1 ~]# timedatectl set-timezone Indian/Mauritius [root@server1 ~]# Managing Time Zone Settings # Between Linux servers, time is normally communicated in UTC. This allows servers located in different time zones to use the same time settings, making it easier to manage large organizations. To make it easier for end users, we should set the local time, and for this we would need to configure an appropriate time zone.\nThere are 3 approaches to setting the local time zone.\nUse timedatectl set-timezone Use the tzselect command to start an text based interface. Go the the /usr/share/zoneinfo directory where you\u0026rsquo;ll find different subdirectories containing files for each time zone. To select a time zone, you create a symbolic link with the name /etc/localtime to the relevant time zone file. e.g. ln -sf /usr/share/zoneinfo/America/Los_Angeles /etc/localtime Configuring Time Service Clients # By default, the chrony service is configured to get the right time from the Internet. In a corporate environment it is not always desirable for clients to go out to the Internet, and instead time servers on the local network are configured.\nIn the below example we\u0026rsquo;ll configure an NTP server on server2 and we\u0026rsquo;ll configure server1 as the client.\nOn server1 we comment out the predefined NTP server in /etc/chrony.conf and define the server2 pool:\n# Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). #pool 2.rhel.pool.ntp.org iburst pool server2 On server2 we edit /etc/chrony.conf to allow connections from a specific subnet, we set a stratum value, then configure the firewall and restart the chronyd service:\n# Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). #pool 2.rhel.pool.ntp.org iburst allow 192.168.0.0/16 local stratum 8 [root@server2 ~]# firewall-cmd --add-service=ntp --permanent success [root@server2 ~]# firewall-cmd --reload success [root@server2 ~]# systemctl restart chronyd [root@server2 ~]# Restart the chronyd service on server1 and check if server2 is used as a source:\n[root@server1 ~]# systemctl restart chronyd [root@server1 ~]# chronyc sources 210 Number of sources = 1 MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== ^? server2 8 6 1 6 +15us[ +15us] +/- 98us [root@server1 ~]# ","date":"15 September 2022","externalUrl":null,"permalink":"/configuring-and-managing-time-services/","section":"Blog","summary":"","title":"Configuring and Managing Time Services","type":"posts"},{"content":"","date":"15 September 2022","externalUrl":null,"permalink":"/tags/hardware-clock/","section":"Tags","summary":"","title":"Hardware Clock","type":"tags"},{"content":"","date":"15 September 2022","externalUrl":null,"permalink":"/tags/network/","section":"Tags","summary":"","title":"Network","type":"tags"},{"content":"","date":"15 September 2022","externalUrl":null,"permalink":"/tags/network-time/","section":"Tags","summary":"","title":"Network Time","type":"tags"},{"content":"","date":"15 September 2022","externalUrl":null,"permalink":"/tags/ntp/","section":"Tags","summary":"","title":"NTP","type":"tags"},{"content":"","date":"15 September 2022","externalUrl":null,"permalink":"/tags/system-time/","section":"Tags","summary":"","title":"System Time","type":"tags"},{"content":"","date":"15 September 2022","externalUrl":null,"permalink":"/tags/time-services/","section":"Tags","summary":"","title":"Time Services","type":"tags"},{"content":"","date":"13 June 2022","externalUrl":null,"permalink":"/tags/cifs/","section":"Tags","summary":"","title":"CIFS","type":"tags"},{"content":" Using NFS Services # The Network File System is a protocol that was developed for UNIX by Sun in the early 1980s. Its purpose is to make mounting of remote file systems in the local file system hierarchy possible. It was often used with Network Information Services (NIS) which provides network-based authentication, all machines connected to the NIS server used the same user accounts and security was handled by the NIS server. NFS security by default is limited to allowing and restricting specific hosts.\nWithout NIS, NFS seems to be an unsecure solution: if on server1 the user X has UID 1001 and on server2 user Y has UID 1001, then user X would have the same access to server2 resources as user Y. To prevent situations like this, NFS should be used together with a centralized authentication service like the Lightweight Directory Access Protocol (LDAP) and Kerberos. This solution is not covered in this article.\nOn RHEL8, NFSv4 is the default version of NFS wich you can override when mounting using the nfsvers= mount option. Typically, clients will automatically fallback to a previous version of NFS if required.\nOffering an NFS Share # To setup an NFS share you would need to go through a few tasks:\nCreate local directories which you want to share and copy some data into them: [root@server2 ~]# mkdir -p /nfs_data /nfs_users/user{1..2} [root@server2 ~]# cp -r /etc/[a-c]* /nfs_data/ [root@server2 ~]# cp -r /etc/[d-f]* /nfs_users/user1/ [root@server2 ~]# cp -r /etc/[g-i]* /nfs_users/user2/ Edit the /etc/exports file to define the NFS shares: [root@server2 ~]# cat /etc/exports /nfs_data\t*(rw,no_root_squash) /nfs_users\t*(rw,no_root_squash) Start and enable the NFS server: [root@server2 ~]# yum install nfs-utils [root@server2 ~]# systemctl enable --now nfs-server Configure the firewall to allow incoming NFS traffic [root@server2 ~]# firewall-cmd --add-service=nfs --permanent success [root@server2 ~]# firewall-cmd --add-service=rpc-bind --permanent success [root@server2 ~]# firewall-cmd --add-service=mountd --permanent success [root@server2 ~]# firewall-cmd --reload success Mounting the NFS Share # In order to mount an NFS share we need to know the name of the share. Typically this information is known by the administrator, but you have multiple options to discover what shares are available:\nIf NFSv4 is used on the server, you can use a root mount. You mount the root directory of the NFS server and you\u0026rsquo;ll see all shares you have access to under your local mount point. Use the showmount -e command The showmount command may have issues with NFSv4 servers that are behind a firewall. The command relies on the portmapper service which uses random UDP ports while the firwall nfs service opens port 2049 only, which doesn\u0026rsquo;t allow portmapper traffic. In these cases you can use the root mount option to discover the shares.\n[root@server1 ~]# showmount -e server2 Export list for server2: /nfs_data * /nfs_users * [root@server1 ~]# mount server2:/ /mnt [root@server1 ~]# ls /mnt/ nfs_data nfs_users Using CIFS Services # Microsoft published the technical specifications of its Server Message Block (SMB) protocol. This protocol is the foundation of all shares that are created in a Windows environment. Releasing these specifications led to the start of the Samba project. The goal of this project was to provide SMB services on top of other operating systems. Samba has developed into the standard for file sharing between different operating systems and is now often referred to as the Common Internet File System (CIFS).\nSetting Up a Samba Server # Before jumping into configuring the samba server, let\u0026rsquo;s clearly define our goals. Server2, the samba server, should be sharing the following directories:\n/var/samba/public_read_share - read only access for guests, mounted on /mnt/public_read_share /var/samba/public_write_share - read/write permissions for guests, mounted on /mnt/public_write_share /var/samba/student_share - read permissions for guests, read/write permissions for users in the students group. Mounted on /mnt/students_share. Installing and Configuring Samba # Install the samba package and create the shared directories:\n[root@server2 ~]# yum install samba -y ... [root@server2 ~]# mkdir -p /var/samba/{public_share,public_write_share,students_share} [root@server2 ~]# ls /var/samba/ public_share public_write_share students_share We enable the smbd_anon_write SELinux Boolean which allows anonymous users to modify public files labeled with the public_content_rw_t file context. Next, we set the appropriate SELinux file contexts:\npublic_content_t - Allows Read Only access to public files. public_content_rw_t - Allows Read/Write access to public files. samba_share_t - As samba doesn\u0026rsquo;t have default paths for shares, we make sure SELinux recognizes our share as a standard samba share. [root@server2 samba]# pwd /var/samba [root@server2 samba]# ls -lh total 0 drwxr-xr-x. 2 root root 6 Mar 11 13:24 public_share drwxr-xr-x. 2 root root 6 Mar 11 13:24 public_write_share drwxr-xr-x. 2 root root 6 Mar 11 13:24 students_share [root@server2 samba]# setsebool -P smbd_anon_write on [root@server2 samba]# getsebool smbd_anon_write smbd_anon_write --\u0026gt; on [root@server2 samba]# semanage fcontext -a -t public_content_t \u0026#34;/var/samba/public_share(/.*)?\u0026#34; [root@server2 samba]# semanage fcontext -a -t public_content_rw_t \u0026#34;/var/samba/public_write_share(/.*)?\u0026#34; [root@server2 samba]# semanage fcontext -a -t samba_share_t \u0026#34;/var/samba/students_share(/.*)?\u0026#34; [root@server2 samba]# restorecon -Rv /var/samba/ Relabeled /var/samba/public_share from unconfined_u:object_r:var_t:s0 to unconfined_u:object_r:public_content_t:s0 Relabeled /var/samba/public_write_share from unconfined_u:object_r:var_t:s0 to unconfined_u:object_r:public_content_rw_t:s0 Relabeled /var/samba/students_share from unconfined_u:object_r:var_t:s0 to unconfined_u:object_r:samba_share_t:s0 [root@server2 samba]# ls -lhZ total 0 drwxr-xr-x. 2 root root unconfined_u:object_r:public_content_t:s0 6 Mar 11 13:24 public_share drwxr-xr-x. 2 root root unconfined_u:object_r:public_content_rw_t:s0 6 Mar 11 13:24 public_write_share drwxr-xr-x. 2 root root unconfined_u:object_r:samba_share_t:s0 6 Mar 11 13:24 students_share Create the students group and, add the user student to the group. Create the smb_user through which we\u0026rsquo;ll be able to write to the public_write_share directory. Add the Linux user student to samba and set a password. This credential will be used to authenticate and mount the students_share directory. Set the Linux permissions on the shared directories:\n[root@server2 samba]# groupadd students [root@server2 samba]# usermod -aG students student [root@server2 samba]# id student uid=1000(student) gid=1000(student) groups=1000(student),1001(students) [root@server2 samba]# useradd smb_user --no-create-home --shell /sbin/nologin [root@server2 samba]# [root@server2 samba]# smbpasswd -a student New SMB password: Retype new SMB password: Added user student. [root@server2 samba]# chgrp smb_user public_write_share [root@server2 samba]# chmod 0770 public_write_share [root@server2 samba]# chmod g+s public_write_share [root@server2 samba]# [root@server2 samba]# chgrp students students_share [root@server2 samba]# chmod 0775 students_share [root@server2 samba]# chmod g+s students_share [root@server2 samba]# [root@server2 samba]# ls -lhZ total 0 drwxr-xr-x. 2 root root unconfined_u:object_r:public_content_t:s0 6 Mar 11 13:24 public_share drwxrws---. 2 root smb_user unconfined_u:object_r:public_content_rw_t:s0 6 Mar 11 13:24 public_write_share drwxrwsr-x. 2 root students unconfined_u:object_r:samba_share_t:s0 6 Mar 11 13:24 students_share Note that we don\u0026rsquo;t change any permissions on public_share, since we only need read access.\nNext, we configure the samba shares in /etc/samba/smb.conf:\n[root@server2 samba]# cd /etc/samba [root@server2 samba]# mv smb.conf smb.conf.old [root@server2 samba]# vim smb.conf ... [root@server2 samba]# testparm Load smb config files from /etc/samba/smb.conf Loaded services file OK. Server role: ROLE_STANDALONE Press enter to see a dump of your service definitions # Global parameters [global] security = USER workgroup = SAMBA idmap config * : backend = tdb [public_read] comment = Public Read Only Share guest ok = Yes path = /var/samba/public_share [public_write] comment = Public Read/Write Share force user = smb_user guest ok = Yes path = /var/samba/public_write_share read only = No write list = smb_user [students] comment = Read/Write access for the students group. Read access for anyone else. guest ok = Yes path = /var/samba/students_share write list = +students We need to allow samba traffic through our firewall:\n[root@server2 samba]# firewall-cmd --add-service=samba --permanent success [root@server2 samba]# firewall-cmd --reload success The final step before moving on to the client side would be to start and enable the samba service:\n[root@server2 samba]# systemctl enable --now smb Created symlink /etc/systemd/system/multi-user.target.wants/smb.service → /usr/lib/systemd/system/smb.service. Discovering CIFS Shares # On server1, where the shares are going to be mounted, you discover available shares using the smbclient -L //hostname command. Make sure you have the cifs-utils and samba-client packages installed:\n[root@server1 ~]# yum install -y cifs-utils samba-client ... Let\u0026rsquo;s discover the shares we created on server2. When you\u0026rsquo;re prompted for a password, just hit Enter without providing a password.\n[root@server1 ~]# smbclient -L //server2 Enter SAMBA\\root\u0026#39;s password: Anonymous login successful Sharename Type Comment --------- ---- ------- public_read Disk Public Read Only Share public_write Disk Public Read/Write Share students Disk Read/Write access for the students group. Read access for anyone else. IPC$ IPC IPC Service (Samba 4.12.3) SMB1 disabled -- no workgroup available [root@server1 ~]# We\u0026rsquo;re ready to move to the next step and mount our shares.\nMounting and Authenticating to Samba Shares # In the previous steps we created two guest shares and one share that needs authentication. We can mount these as follows:\nmount -t cifs -o guest //server2/public_read /mnt/public_read_share mount -t cifs -o guest //server2/public_write /mnt/public_write_share mount -t cifs -o username=student,password=password //server2/students_share /mnt/students_share Before you do so, create the local mount points:\n[root@server1 ~]# mkdir /mnt/{public_read_share,public_write_share,students_share} [root@server1 ~]# ls -l /mnt/ total 0 drwxr-xr-x. 2 root root 6 Mar 11 14:41 public_read_share drwxr-xr-x. 2 root root 6 Mar 11 14:41 public_write_share drwxr-xr-x. 2 root root 6 Mar 11 14:41 students_share [root@server1 ~]# mount -t cifs -o guest //server2/public_read /mnt/public_read_share [root@server1 ~]# mount -t cifs -o guest //server2/public_write /mnt/public_write_share [root@server1 ~]# mount -t cifs -o username=student,password=password //server2/students /mnt/students_share Next, test the read/write access to the shares. The outcome should be as expected.\nNote that we\u0026rsquo;ve mounted the share as root, this means the /mnt/students_share directory will only be writeable for the user root. In the next step we\u0026rsquo;ll cover how to auto mount the share at boot time.\nMounting Remote File Systems Through fstab # As we\u0026rsquo;ve seen in an earlier post, the /etc/fstab file can be used to mount file systems automatically at boot time.\nMounting NFS Shares Through fstab # Mounting NFS Shares through /etc/fstab is pretty straightforward. Add the following line to the fstab file:\nserver2:/nfs_data\t/nfs_data\tnfs sync 0 0 With the sync option we ensure that modified files are committed to the remote file system immediately instead of being placed in a write buffer.\nMounting Samba Shares Through fstab # When mounting Samba file systems through /etc/fstab, you need to consider a specific challenge: The user credentials that are needed to issue the mount. These are typically specified as mount options using username= and password=, but it is not a good idea to put these in clear text in the /etc/fstab file.\nWe can work around this by creating a file in the root home that contains these credentials, and referencing /etc/fstab to that file:\n[root@server1 ~]# pwd /root [root@server1 ~]# cat cifs.txt user=student pass=password [root@server1 ~]# We set strict permissions on the file so only root can read it:\n[root@server1 ~]# chmod 0600 cifs.txt [root@server1 ~]# Next, for the //server2/students share, we add the following line to /etc/fstab:\n//server2/students\t/mnt/students_share\tcifs\tcredentials=/root/cifs.txt,gid=students,file_mode=0664,dir_mode=0775 0 0 Let\u0026rsquo;s break down what the line does exactly:\n//server2/students - The remote file system we\u0026rsquo;re mounting. /mnt_students_share - The local mount point of the share. cifs - The remote file system type. credentials=/root/cifs.txt - Specifies the file that contains the credentials necessary to mount the remote file system. gid=students - We set group ownership on the files and directories to the group students . file_mode=0664 - We set the necessary file permissions: read+write for Owner and Group, read for Others. dir_mode=0775 - We set the necessary directory permissions: read+write+execute for Owner and Group, read+execute for Others. 0 0 - We don\u0026rsquo;t need backup support through the dump utility and we don\u0026rsquo;t want fsck to check the disk integrity during boot. Simarly to the above, the entry for the //server2/public_write_share would look like this:\n//server2/public_write_share\t/mnt/public_write_share cifs\tguest,file_mode=0777,dir_mode=0777\t0 0 We autenticate as the user guest aganst the remote file system and we allow everyone read+write access to files and, read+write+execute permissions to directories.\nFor the last share, //server2/public_share, we don\u0026rsquo;t specify Linux permissions in the /etc/fstab file as this share has been set to read-only by default on the Samba server.\n//server2/public_share\t/mnt/public_read_share\tcifs\tguest\t0 0 Here all three /etc/fstab entries:\n//server2/public_share\t/mnt/public_read_share\tcifs\tguest\t0 0 //server2/public_write_share\t/mnt/public_write_share cifs\tguest,file_mode=0666,dir_mode=0777\t0 0 //server2/students\t/mnt/students_share\tcifs\tcredentials=/root/cifs.txt,gid=students,file_mode=0664,dir_mode=0775 0 0 Using Automount to Mount Remote File Systems # As an alternative to using /etc/fstab we can configure automount to mount the shares automatically. The difference is that mounts through automount are \u0026ldquo;on demand\u0026rdquo;, which ensures that no files systems are mounted when it\u0026rsquo;s not needed. This works completely in user space and no root permissions are required, contrary to mounts using the mount command.\nYou need to install the autofs package to use automount:\n[root@server1 ~]# yum install -y autofs ... [root@server1 ~]# systemctl enable --now autofs ... Defining Mounts in Automount # Mounts in automount are defined through a two-step procedure:\nEdit the master configuration file in /etc/auto.master where you specify the local mount point and the secondary configuration file. Edit the secondary configuration file where you specify the subdirectory that will be created in the mount point. For this exercise, we\u0026rsquo;ll be using the nfs_data NFS share on server2:\n[root@server2 ~]# cat /etc/exports /nfs_users\t*(rw,no_root_squash) /nfs_data\t*(rw,no_root_squash) On server1, open the /etc/auto.master file and add the below line:\n/nfs_data\t/etc/auto.nfs_data On server1, open the /etc/auto.nfs_data file and add the below line:\nfiles -rw server2:/nfs_data Restart the autofs service:\n[root@server1 /]# systemctl restart autofs Go to the /nfs_data directory on server1, notice there is no files directory:\n[root@server1 nfs_data]# ls [root@server1 nfs_data]# Change directory to /nfs_data/files:\n[root@server1 nfs_data]# cd files [root@server1 files]# ls automount_test The /nfs_data share on server2 was auto mounted on /nfs_data/files on server1.\nUsing Wildcards in Automount # In some cases we\u0026rsquo;re better off using dynamic directory names, for example when mounting home directories. The home directory of a user would be automatically mounted when that user logs in.\nWe\u0026rsquo;ll be simulating this by using the /nfs_users NFS share on server2:\n[root@server2 ~]# cat /etc/exports /nfs_users\t*(rw,no_root_squash) /nfs_data\t*(rw,no_root_squash) First, unmount the /nfs_users mount point on server1, if you still have it mounted, and delete the directory:\n[root@server1 /]# umount /nfs_users [root@server1 /]# rm -rf nfs_users Add the below line to the /etc/auto.master file on server1:\n/nfs_users /etc/auto.nfs_users Create the /etc/auto.nfs_users file and add the below:\n* -rw server2:/nfs_users/\u0026amp;` Restart the autofs service:\n[root@server1 /]# systemctl restart autofs Go to the /nfs_users directory and notice it\u0026rsquo;s empty:\n[root@server1 /]# cd /nfs_users [root@server1 nfs_users]# ls [root@server1 nfs_users]# Change directory to /nfs_users/user1:\n[root@server1 nfs_users]# cd user1 [root@server1 user1]# ls user1_automount_test [root@server1 user1]# See how the other user folders are auto-mounted on demand:\n[root@server1 nfs_users]# ls user1 [root@server1 nfs_users]# cd user2 [root@server1 user2]# ls user2_automount_test [root@server1 user2]# cd .. [root@server1 nfs_users]# ls user1 user2 ","date":"13 June 2022","externalUrl":null,"permalink":"/configuring-and-auto-mounting-remote-file-systems-using-fstab-and-automount-NFS-CIFS/","section":"Blog","summary":"","title":"Configuring and Auto Mounting Remote File Systems Using fstab and automount: NFS \u0026 CIFS","type":"posts"},{"content":"","date":"13 June 2022","externalUrl":null,"permalink":"/tags/network-storage/","section":"Tags","summary":"","title":"Network Storage","type":"tags"},{"content":"","date":"13 June 2022","externalUrl":null,"permalink":"/tags/nfs/","section":"Tags","summary":"","title":"NFS","type":"tags"},{"content":"","date":"13 June 2022","externalUrl":null,"permalink":"/tags/samba/","section":"Tags","summary":"","title":"Samba","type":"tags"},{"content":"","date":"1 May 2022","externalUrl":null,"permalink":"/tags/firewall/","section":"Tags","summary":"","title":"Firewall","type":"tags"},{"content":"","date":"1 May 2022","externalUrl":null,"permalink":"/tags/firewalld-service/","section":"Tags","summary":"","title":"Firewalld Service","type":"tags"},{"content":" Understanding Linux Firewalling # Firewalling is implemented in the Linux kernel by means of the netfilter subsystem to limit traffic coming in to a server or going out of the server. Netfilter allows kernel modules to inspect every incoming, outgoing, or forwarded packet and act upon it by either allowing it or blocking it. In essence, netfilter controls access to and from the network stack at the Linux kernel module level.\nIptables was the default solution to interact with netfilter, it provides a sophisticated way of defining firewall rules but it\u0026rsquo;s also challenging to use due to the complicated syntax and the ordering of rules which can become complex. The iptables service is no longer offered in RHEL8, it has been replaced with nftables, a new solution with more advanced options.\nFirewalld # Firewalld is a higher-level netfilter implementation that is more user-friendly compared to iptables or nftables. While administrators can manage the Firewalld rules, applications can also communicate with it using the DBus messaging system: rules can be added or removed without any direct action required from the system administrator. Applications can address the firewall from user space.\nFirewalld applies rules to incoming packets only by default, no filtering happens on outgoing packets.\nFirewalld Zones # Firewalld makes management easier by working with zones. A zone is a collection of rules that are applied to incoming packets matching a specific source address or network interface.\nThe use of zones is import on servers that have multiple network interfaces. Each interface could be a different zone where different rules would apply. On a machine with only one network interface you can work with one zone, the default zone.\nEvery packet that comes into a system is analyzed for its source address, based on the source address Firewalld decides if it belongs to a specific zone. If not, the zone for the incoming network interface is used. If no specific zone is available, the packet is handled by the rules in the default zone.\nZone Name Description block Incoming network connections are rejected with the \u0026ldquo;icmp-host-prohibited\u0026rdquo; message. Connections that were initiated on this system are allowed. dmz For use on computers in the demilitarized zone. Selected incoming connections are accepted, and limited access to the internal network is allowed. drop Any incoming packets are dropped and there is no reply. external For use on external networks with masquarading (Network Address Translation) enabled, used on routers. Selected incoming connections are accepted. home Most computers on the same network are trusted, only selected incoming connections are accepted. internal Most computers on the same network are trusted, only selected incoming connections are accepted. public Other computers on the same network are not trused, limited connections are accepted. This is the default zone for all newly created network interfaces. trusted All network connections are accepted. work Most computers on the same network are trusted, only selected incoming connections are accepted. Firewalld Services # Services are the second key element while working with Firewalld. A service in Firewalld is not the same as a service in systemd. A Firewalld service defines what exactly should be accepted as incoming traffic in the firewall, it includes ports to be opened and supoorting kernel modules that should be loaded.\nBehind each service is an XML configuration file that explains which TCP or UDP ports are involved and, if required, what kernel modules must be loaded. Default (RPM installed) XML files are stored in /usr/lib/firewalld/services while custom XML files can be added to the /etc/firewalld/services directory.\n[root@localhost ~]# firewall-cmd --get-services RH-Satellite-6 amanda-client amanda-k5-client amqp amqps ... ... [root@localhost ~]# cat /usr/lib/firewalld/services/ftp.xml \u0026lt;?xml version=\u0026#34;1.0\u0026#34; encoding=\u0026#34;utf-8\u0026#34;?\u0026gt; \u0026lt;service\u0026gt; \u0026lt;short\u0026gt;FTP\u0026lt;/short\u0026gt; \u0026lt;description\u0026gt;FTP is a protocol used for remote file transfer. If you plan to make your FTP server publicly available, enable this option. You need the vsftpd package installed for this option to be useful.\u0026lt;/description\u0026gt; \u0026lt;port protocol=\u0026#34;tcp\u0026#34; port=\u0026#34;21\u0026#34;/\u0026gt; \u0026lt;helper name=\u0026#34;ftp\u0026#34;/\u0026gt; \u0026lt;/service\u0026gt; Working with Firewalld # Firewalld provides a command-line interface tool that works with a runtime and permament (on-disk) configuration state: firewall-cmd\nBelow is an example of how you can use the tool to retrieve current settings and make configuration changes. Always make sure to commit changes to disk using the --permanent flag so that your changes survive a reboot, then --reload to apply the changes to the runtime environment.\n[root@localhost ~]# firewall-cmd --get-default-zone public [root@localhost ~]# firewall-cmd --get-zones block dmz drop external home internal libvirt public trusted work [root@localhost ~]# firewall-cmd --list-all --zone=public public (active) target: default icmp-block-inversion: no interfaces: enp1s0 sources: services: cockpit dhcpv6-client ftp http https ssh ports: protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: [root@localhost ~]# firewall-cmd --get-services RH-Satellite-6 amanda-client amanda-k5-client amqp amqps apcupsd audit bacula bacula-client bb bgp bitcoin bitcoin-rpc bitcoin-testnet ... [root@localhost ~]# firewall-cmd --list-services cockpit dhcpv6-client ftp http https ssh [root@localhost ~]# firewall-cmd --add-service=vnc-server --permanent success [root@localhost ~]# firewall-cmd --list-services cockpit dhcpv6-client ftp http https ssh [root@localhost ~]# firewall-cmd --reload success [root@localhost ~]# firewall-cmd --list-services cockpit dhcpv6-client ftp http https ssh vnc-server [root@localhost ~]# firewall-cmd --add-port=2022/tcp --permanent success [root@localhost ~]# firewall-cmd --reload success [root@localhost ~]# firewall-cmd --list-all public (active) target: default icmp-block-inversion: no interfaces: enp1s0 sources: services: cockpit dhcpv6-client ftp http https ssh vnc-server ports: 2022/tcp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: Key Commands # firewall-cmd --list-all firewall-cmd --list-all --zone=public firewall-cmd --get-default-zone firewall-cmd --get-zones firewall-cmd --get-services firewall-cmd --list-services firewall-cmd --add-service ftp irewall-cmd --add-service ftp --permanent firewall-cmd --reload firewall-cmd --add-port=2022/tcp --permanent firewall-cmd --reload ","date":"1 May 2022","externalUrl":null,"permalink":"/managing-a-firewall-with-firewalld/","section":"Blog","summary":"","title":"Managing a Firewall with Firewalld","type":"posts"},{"content":"","date":"1 May 2022","externalUrl":null,"permalink":"/tags/security/","section":"Tags","summary":"","title":"Security","type":"tags"},{"content":" SELinux is a security enhancement module, deployed on top of Linux, which provides improved security via Role Based Access Controls (RBACs) on subjects and objects (processes and resources). Traditional Linux security used Discretionary Access Controls (DACs).\nWith DAC, a process can access any file, directory, device or other resource that leaves itself open to access. Using RBAC, a process only has access to resources that it is explicitely allowd to access, based on the assigned role. The way that SELinux implements RBAC is to assign an SELinux policy to a process. That process restricts access as follows:\nOnly let the process access resources that carry the explicit labels Make potentially insecure features, e.g. write access to a directory, available as Booleans, which can be turned on or off. SELinux is not a replacement for DAC, it\u0026rsquo;s an additional security layer:\nDAC rules are still used when using SELinux; DAC rules are checked first, if those allow access then SELinux policies are checked; If DAC rules deny access then SELinux policies are not checked. In essence, SELinux severaly limits what potentially malicious code may gain access to and generally limits activity on the Linux system.\nUnderstanding How SELinux Works # SELinux provides a combination of Role Based Access Control and either Type Enforcement (TE) or Multi-Level Security (MLS). In RBAC, access to an object is granted or denied based on the subject\u0026rsquo;s assigned role in the organization. It\u0026rsquo;s not based on usernames or process ID. In this post I will focus only on Type Enforcement, which is the default SELinux targeted policy.\nType Enforcement # Type Enforcement is necessary to implement the RBAC model, it secures a system through these methods:\nLabeling objects as certain security types; Assigning subjects to particular domains and roles; Providing rules to allow certain domains and roles to access certain object types. Let\u0026rsquo;s look at an example. The below ls -l command shows the DAC controls on the files. The output shows the file\u0026rsquo;s owner, group and permissions:\n[student@localhost my_stuff]$ ls -l total 0 -rw-rw-r--. 1 student student 0 Jan 19 06:25 test001 We can add the -Z option to display the SELinux RBAC controls too:\n[student@localhost my_stuff]$ ls -lZ total 0 -rw-rw-r--. 1 student student unconfined_u:object_r:user_home_t:s0 0 Jan 19 06:25 test001 The last example displays four items assiciated with the file that are specific to SELinux:\nuser unconfined_u role object_r type user_home_t level s0 The above four RBAC items are used in the SELinux access control to determine appropriate access levels. Together, these items are called the SELinux security context or sometimes the security label.\nThese security contexts are given to to subjects (processes and users). Each security context has a specific name. The name given depends upon what object or subject it has been assigned: Files have a file context, users have a user context, and processes have a process context also referred to as a domain.\nThe rules allowing access are called allow rules or policy rules. A policy rule is the process SELinux follows to grant or deny access to a particular system security type. Thus, Type Enforcement ensures that only certain \u0026ldquo;types\u0026rdquo; of subjects can access certain \u0026ldquo;types\u0026rdquo; of objects.\nImplementing SELinux Security Models # SELinux implements the RBAC model through a combination of four primary SELinux pieces:\nOperational modes Security contexts Policy types Policy rule packages We already touched on some of these design elements.\nUnderstanding SELinux Operational Modes # SELinux comes with three operational modes: disabled, permissive and enforcing. Each of these modes offeres different benefits for Linux system security.\nUsing Disabled Mode # In the disabled mode, SELinux is turned off. The default method of access control, Discretionary Access Control, is used instead.\nUsing Permissive Mode # In permissive mode, SELinux is turned on, but the security policy rules are not enforced. When a security policy rule should deny access, access will still be allowed. However, a message is sent to a log file denoting that access should\u0026rsquo;ve been denied.\nSELinux permissive mode is useful for testing and troubleshooting.\nUsing Enforcing Mode # In enforcing mode SELinux is turned on and all of the security policy rules are enforced.\nUnderstanding SELinux Security Contexts # An SELinux security context is the method used to classify objects (such as files) and subjects (such as users or programs). A security context consists of four attributes: user, role, type and level.\nUser - The user attribute is a mapping of a Linux username to an SELinux name. This is not the same as a users\u0026rsquo;s login name, and it\u0026rsquo;s referred to specifically as the SELinux user. The SELinux username ends with a _u, making it easy to identify in the output. Regular unconfined users have an unconfined_u user attribute in the default targeted policy.\nRole - The role attribute is assigned to subjects and objects. Each role is granted access to other subjects and objects based on the role\u0026rsquo;s security clearance and the object\u0026rsquo;s classification level. Users are assigned a role and that role is authorized for particular types of domains (or process context). The SELinux role has _r at the end. Processes run by root have a system_r role, while regular users run processes under the unconfined_r role.\nType - The type attribute defines a domain type for processes, a user type for users, and a file type for files. This attribute is also called the security type. Most policy rules are concerned with the security type of a process and what files, ports, devices and other resources that process has access to based on their security types. The SELinux type name ends with a _t.\nLevel - The level is an attribute of Multi-Level Security (MLS), it\u0026rsquo;s optional in Type Enforcement.\nUsers, Files, and Processes Have Security Contexts # To see your SELinux user context, enter the id command at the shell prompt:\n[student@localhost ~]$ id uid=1000(student) gid=1000(student) groups=1000(student) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 [student@localhost ~]$ Use the -Z option on the ls command to see an individual file\u0026rsquo;s context:\n[student@localhost my_stuff]$ ls -lZ total 0 -rw-rw-r--. 1 student student unconfined_u:object_r:user_home_t:s0 0 Jan 19 06:25 test001 Use the -Z option on the ps command to see a process\u0026rsquo;s security context:\n[student@localhost my_stuff]$ ps -eZ | grep bash unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 2872 pts/0 00:00:00 bash [student@localhost my_stuff]$ ps -eZ | grep systemd system_u:system_r:init_t:s0 1 ? 00:00:01 systemd system_u:system_r:syslogd_t:s0 638 ? 00:00:00 systemd-journal Understanding SELinux Policy Types # The policy type directly determines what sets of policy rules are used to dictate what an object can access. The policy type also determines what specific security context attributes are needed.\nSELinux has different policies:\nTargeted (default) MLS Minimum The Targeted policy\u0026rsquo;s primary purpose is to restrict \u0026ldquo;targeted\u0026rdquo; daemons, but it can also restrict other processes and users. Targeted daemons are sandboxed, they run in an environment where their access to other objects is tightly controlled so that no malicious attacks launched through those daemons can affect other services or the Linux system as a whole.\nAll subjects and objects not targeted are run in the unconfined_t domain. This domain has no SELinux policy restrictions and thus only used traditional Linux security.\nSELinux Policy Rule Packages # Policy rules are installed with SELinux and are grouped into packages, also called modules.\nThere is user documentation on these various policy modules in the form of HTML files. To view this documentation on RHEL, open your browser and enter the following url: file:///usr/share/doc/selinux-policy/html/index.html\nIf you don\u0026rsquo;t have the policy documentation you can install it: yum install selinux-policy-doc\nThis documentation allows you to review how policy rules are created and packaged.\nConfiguring SELinux # SELinux comes preconfigured, you can use the SELinux features without any configuration. The configuration can only be set and modified by root. The primary configuration file is /etc/sysconfig/selinux which is a symlink to /etc/selinux/config:\n[root@localhost ~]# ls -lh /etc/sysconfig/selinux lrwxrwxrwx. 1 root root 17 Sep 26 09:44 /etc/sysconfig/selinux -\u0026gt; ../selinux/config [root@localhost ~]# cat /etc/sysconfig/selinux # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=enforcing # SELINUXTYPE= can take one of these three values: # targeted - Targeted processes are protected, # minimum - Modification of targeted policy. Only selected processes are protected. # mls - Multi Level Security protection. SELINUXTYPE=targeted This file allows you to set the mode and policy type.\nSetting the SELinux Mode and Policy Type # We can use the getenforce command to see the current SELinux mode, to see both the current mode and the mode set in the configuration file, use the sestatus command:\n[root@localhost ~]# getenforce Enforcing [root@localhost ~]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: enforcing Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Memory protection checking: actual (secure) Max kernel policy version: 31 To change the mode setting, you can use the setenforce command with either the 0 or permissive argument, or the 1 or enforcing argument. This will change the SELinux mode during runtime and leaves the setting in the primary configuration file untouched. Rebooting the system will apply the mode set in the primary configuration file.\nYou cannot use setenforce to change SELinux to disabled mode.\nSwitching from disabled to enforcing should be done using the primary configuration file and a reboot. Using the setenforce command may hang your system due to incorrect file labels. Rebooting after changing from disabled may take a while as the filesystem will be relabeled.\nThis means that SELinux checks and changes the security context of any files with incorrect security contexts that can cause problems in the new mode, and any file not labeled will be labeled with contexts. This process can take a long time since each file\u0026rsquo;s context is checked.\nThe policy type you choose determines whether SELinux enforces TE, MLS or Minimum. The default policy type is targeted. When setting the policy type to MLS or Minimum you need to make sure you have the policy package installed: yum list selinux-policy-mls selinux-policy-minimum\nManaging SELinux Security Contexts # Current SELinux file and process security contexts can be viewed using the secon command:\n-u Shows the user of the security context. -r Shows the role of the security context. -t Shows the type of the security context. Without any arguments, the command shows you the current process\u0026rsquo;s security context:\n[student@localhost ~]$ secon -urt user: unconfined_u role: unconfined_r type: unconfined_t To view another process\u0026rsquo;s security context, use the -p option followed by the process id. e.g. systemd:\n[student@localhost ~]$ secon -urt -p 1 user: system_u role: system_r type: init_t To view a file\u0026rsquo;s security context, use the -f option:\n[student@localhost ~]$ secon -urt -f /etc/passwd user: system_u role: object_r type: passwd_file_t The secon command does not show the security context for the current user, instead use the id command.\nSetting Security Context Types # Since the RHCSA exam focuses only on context types, I will not be covering the user and role contexts.\nTo set a context type we can use the semanage command. semanage writes the new context to the SELinux policy from where it can be applied to the file system.\nThe semanage command may not be available by default. You can find the RPM containing semanage using yum whatprovides */semanage:\n[root@localhost ~]# yum whatprovides */semanage policycoreutils-python-utils-2.9-9.el8.noarch : SELinux policy core python utilities Repo : BaseOS Matched from: Filename : /usr/sbin/semanage Filename : /usr/share/bash-completion/completions/semanage The policycoreutils-python-utils has to be installed in order to use semanage.\nTo set context using semanage we need to know the appropriate context type. An easy way to find the appropriate context is by looking at the default context settings on already-existing items:\n[root@localhost ~]# ls -lZ /var/www total 0 drwxr-xr-x. 2 root root system_u:object_r:httpd_sys_script_exec_t:s0 6 Jun 8 2020 cgi-bin drwxr-xr-x. 4 root root system_u:object_r:httpd_sys_content_t:s0 61 Jan 6 12:19 html /var/www/html is a default location for the Apache HTTP Service. If we would want to add a new folder to /var/www to serve content with Apache, we now know we need the http_sys_content_t context type.\nFor demonstration purposes, let\u0026rsquo;s created the my_dir directory in our home folder, then move it to /var/www. The reason why we do this is because if we create the directory in /var/www it will inherit the correct context type from the parent directory.\n[root@localhost ~]# ls -lZ /var/www total 0 drwxr-xr-x. 2 root root system_u:object_r:httpd_sys_script_exec_t:s0 6 Jun 8 2020 cgi-bin drwxr-xr-x. 4 root root system_u:object_r:httpd_sys_content_t:s0 61 Jan 6 12:19 html drwxr-xr-x. 2 root root unconfined_u:object_r:admin_home_t:s0 6 Jan 20 11:44 my_dir The mv command kept the admin_home_t context type on our directory. We can change the context type as follows:\n[root@localhost ~]# semanage fcontext -a -t httpd_sys_content_t \u0026#34;/var/www/my_dir(/.*)?\u0026#34; [root@localhost ~]# ls -lZd /var/www/my_dir drwxr-xr-x. 2 root root unconfined_u:object_r:admin_home_t:s0 6 Jan 20 11:44 /var/www/my_dir The -a option is used to add a context type, then we use -t to specify the context type. The last part of the command indicates the folder we apply the changes to and contains a regular expression, (/.*)?, to refer to the directory my_dir and anything that exists below that directory.\nNotice how the semanage command didn\u0026rsquo;t provide any output, and our ls -lZd command still shows the original context type. This is because using semanage we only applied the context type to the SELinux policy but not to the file system. We need to apply the change to the file system using restorecon:\n[root@localhost ~]# restorecon -R -v /var/www/my_dir Relabeled /var/www/my_dir from unconfined_u:object_r:admin_home_t:s0 to unconfined_u:object_r:httpd_sys_content_t:s0 [root@localhost ~]# ls -lZd /var/www/my_dir drwxr-xr-x. 2 root root unconfined_u:object_r:httpd_sys_content_t:s0 6 Jan 20 11:44 /var/www/my_dir The following example changes the SELinux context type on a network port, assuming you would want to make the ssh service available over port 2222.\n[root@localhost ~]# semanage port -l | grep ssh ssh_port_t tcp 22 [root@localhost ~]# semanage port -a -t ssh_port_t -p tcp 2222 [root@localhost ~]# semanage port -l | grep ssh ssh_port_t tcp 2222, 22 Finding the Context Type You Need # There are three approaches in finding the context type you need:\nLook at the default environment; Read the configuration files; Use man -k _selinux to find the SELinux-specific man pages for your service. The man pages are not installed by default, to install them you need to install the policycoreutils-devel package. Once installed, use the mandb command to update the man page database and issue the sepolicy manpage -a -p /usr/share/man/man8 command to install the SELinux man pages:\n[root@localhost ~]# yum whatprovides */sepolicy policycoreutils-devel-2.9-9.el8.i686 : SELinux policy core policy devel utilities Repo : BaseOS Matched from: Filename : /usr/bin/sepolicy Filename : /usr/share/bash-completion/completions/sepolicy [root@localhost ~]# yum install -y policycoreutils-devel ... [root@localhost ~]# sepolicy manpage -a -p /usr/share/man/man8 ... [root@localhost ~]# mandb ... [root@localhost ~]# man -k _selinux | grep http apache_selinux (8) - Security Enhanced Linux Policy for the httpd processes httpd_helper_selinux (8) - Security Enhanced Linux Policy for the httpd_helper processes httpd_passwd_selinux (8) - Security Enhanced Linux Policy for the httpd_passwd processes ... [root@localhost ~]# man apache_selinux Restoring Default File Contexts # Previously, we applied the context type from the policy to the file system using the restorecon command. The policy contains the default settings for most files and directories, so if ever a wrong context setting is applied we can use restorecon to reapply the default from the policy to the file system.\nUsing restorecon this way can be useful to fix problems on new files. There\u0026rsquo;s a specific way context settings are applied:\nIf a new file or directory is created, it inherits the context type of the parent directory. If a file or directory is copied, this is considered a new file or directory. If a file is moved, or copied using cp -a and thus keeping properties, the original context type is applied. The latter of the above 4 ways can be fixed by using restorecon. It\u0026rsquo;s also possible to relabel the entire file system using restorecon -Rv / or by creating the file /.autorelabel in the root /. The next time you reboot the system will discover the /.autorelabel file and the entire file system will be relabeled.\nManaging SELinux via Booleans # SELinux Booleans are provided to easily change the behaviour of a rule. A Boolean is a switch that toggles a setting on or off and it allows you to change parts of a SELinux policy rule without any knowledge of policy writing. These changes are applied during runtime and do not require a reboot.\nYou can get a list of Booleans using the getsebool -a command and filtering that down using grep:\n[root@localhost ~]# getsebool -a | grep httpd httpd_anon_write --\u0026gt; off httpd_builtin_scripting --\u0026gt; on httpd_can_check_spam --\u0026gt; off httpd_can_connect_ftp --\u0026gt; off httpd_can_connect_ldap --\u0026gt; off httpd_can_connect_mythtv --\u0026gt; off httpd_can_connect_zabbix --\u0026gt; off ... The semanage boolean -l command provides more detail, it shows the current setting and the default one.\n[root@localhost ~]# semanage boolean -l | head SELinux boolean State Default Description abrt_anon_write (off , off) Allow ABRT to modify public files used for public file transfer services. abrt_handle_event (off , off) Determine whether ABRT can run in the abrt_handle_event_t domain to handle ABRT event scripts. To set a Boolean we use setsebool and to apply the change permanently we add the -P option:\n[root@localhost ~]# getsebool -a | grep ftpd ftpd_anon_write --\u0026gt; off ... [root@localhost ~]# setsebool ftpd_anon_write on [root@localhost ~]# semanage boolean -l | grep ftpd_anon ftpd_anon_write (on , off) Determine whether ftpd can modify public files used for public file transfer services. Directories/Files must be labeled public_content_rw_t. [root@localhost ~]# setsebool -P ftpd_anon_write on [root@localhost ~]# semanage boolean -l | grep ftpd_anon ftpd_anon_write (on , on) Determine whether ftpd can modify public files used for public file transfer services. Directories/Files must be labeled public_content_rw_t. Troubleshooting SELinux Policy Violations # SELinux logs everything it is doing, the primary source to get logging information is the audit log which is in /var/log/audit/audit.log. Message are logged with type=AVC, which stands for Access Vector Cache.\n[root@localhost ~]# grep AVC /var/log/audit/audit.log | tail -1 type=AVC msg=audit(1611246770.937:136): avc: denied { getattr } for pid=4178 comm=\u0026#34;httpd\u0026#34; path=\u0026#34;/test/index.html\u0026#34; dev=\u0026#34;dm-0\u0026#34; ino=35157701 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:default_t:s0 tclass=file permissive=0 The first relevant part in the output is the text acv: denied { getattr }. This means some process tried to read the attributes of a file and it was denied access. Further down we can see comm=\u0026quot;httpd\u0026quot; which means the command that was trying to issue the getattr request was httpd. Next, we see path=\u0026quot;test/index.html\u0026quot;, which is the file that this process tried to access.\nIn the last part we see information about the source context and the target context: scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:default_t:s0\ndefault_t is used for files that do not match any pattern in the SELinux policy. I created /test/index.html in the root and SELinux doesn\u0026rsquo;t know what security context to give to this file, so it assigned default_t.\nWe also see that Permissive mode is disabled: permissive=0\nThe issue here is that the SELinux policy denies access from the httpd_t security context to the default_t security context. We can solve this issue by setting the correct target security context:\n[root@localhost ~]# semanage fcontext -a -t httpd_sys_content_t \u0026#34;/test(/.*)?\u0026#34; [root@localhost ~]# restorecon -Rv /test Relabeled /test from unconfined_u:object_r:default_t:s0 to unconfined_u:object_r:httpd_sys_content_t:s0 Relabeled /test/index.html from unconfined_u:object_r:default_t:s0 to unconfined_u:object_r:httpd_sys_content_t:s0 Analyzing SELinux with Sealert # We can use sealert to easier understand SELinux messages in /var/log/audit/audit.log. First, you need to install sealert: yum install setroubleshoot-server\nOnce this is installed, issue the journalctl | grep sealert command:\nJan 21 11:32:57 localhost.localdomain setroubleshoot[4395]: SELinux is preventing httpd from getattr access on the file /test/index.html. For complete SELinux messages run: sealert -l e4fc58ab-c1c0-4525-a955-eff9a5570a7c Jan 21 11:33:00 localhost.localdomain setroubleshoot[4395]: SELinux is preventing httpd from getattr access on the file /test/index.html. For complete SELinux messages run: sealert -l e4fc58ab-c1c0-4525-a955-eff9a5570a7c Follow the instructions and run saelert -l UUID. sealert will analyze what\u0026rsquo;s happened and provide some suggestions what you need to do to fix the problem. Each suggestion will have a confidence score and the higher this score the more likely the suggested solution would be applicable.\n[root@localhost ~]# sealert -l e4fc58ab-c1c0-4525-a955-eff9a5570a7c SELinux is preventing httpd from getattr access on the file /test/index.html. ***** Plugin catchall_labels (83.8 confidence) suggests ******************* If you want to allow httpd to have getattr access on the index.html file Then you need to change the label on /test/index.html Do # semanage fcontext -a -t FILE_TYPE \u0026#39;/test/index.html\u0026#39; ... ","date":"29 March 2022","externalUrl":null,"permalink":"/enhancing-linux-security-with-selinux/","section":"Blog","summary":"","title":"Enhancing Linux Security with SELinux","type":"posts"},{"content":"","date":"29 March 2022","externalUrl":null,"permalink":"/tags/selinux/","section":"Tags","summary":"","title":"SELinux","type":"tags"},{"content":"","date":"7 December 2021","externalUrl":null,"permalink":"/tags/apache/","section":"Tags","summary":"","title":"Apache","type":"tags"},{"content":"","date":"7 December 2021","externalUrl":null,"permalink":"/tags/httpd/","section":"Tags","summary":"","title":"Httpd","type":"tags"},{"content":"","date":"7 December 2021","externalUrl":null,"permalink":"/tags/network-services/","section":"Tags","summary":"","title":"Network Services","type":"tags"},{"content":" Managing Apache HTTP Services is not part of the current RHCSA exam objectives, but we need minimal knowledge on this topic in order to master the SELinux-related objectives later on.\nThe Apache server is provided through different software packages. The basic packages is httpd which contains everything for an operational but basic website. For a complete overview of all the packages use yum search httpd.\nUnderstanding the httpd Package # Let\u0026rsquo;s examine the httpd package by downloading it using yumdownloader and running a few rpm commands on it:\n[root@localhost ~]# yumdownloader httpd Last metadata expiration check: 0:00:31 ago on Tue 06 Oct 2020 11:05:52 AM EST. [root@localhost ~]# ls anaconda-ks.cfg httpd-2.4.37-21.module_el8.2.0+382+15b0afa8.x86_64.rpm initial-setup-ks.cfg [root@localhost ~]# rpm -qpi httpd-2.4.37-21.module_el8.2.0+382+15b0afa8.x86_64.rpm Name : httpd Version : 2.4.37 Release : 21.module_el8.2.0+382+15b0afa8 Architecture: x86_64 Install Date: (not installed) Group : System Environment/Daemons Size : 5105105 License : ASL 2.0 Signature : RSA/SHA256, Mon 08 Jun 2020 05:08:58 PM EDT, Key ID 05b555b38483c65d Source RPM : httpd-2.4.37-21.module_el8.2.0+382+15b0afa8.src.rpm Build Date : Mon 08 Jun 2020 04:15:29 PM EDT Build Host : x86-02.mbox.centos.org Relocations : (not relocatable) Packager : CentOS Buildsys \u0026lt;bugs@centos.org\u0026gt; Vendor : CentOS URL : https://httpd.apache.org/ Summary : Apache HTTP Server Description : The Apache HTTP Server is a powerful, efficient, and extensible web server. [root@localhost ~]# We can see the package was created by CentOS Buildsys and that it is indeed the Apache HTTP Server package. Next, let\u0026rsquo;s have a look at the configuration files:\n[root@localhost ~]# rpm -qpc httpd-2.4.37-21.module_el8.2.0+382+15b0afa8.x86_64.rpm /etc/httpd/conf.d/autoindex.conf /etc/httpd/conf.d/userdir.conf /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.modules.d/00-base.conf /etc/httpd/conf.modules.d/00-dav.conf /etc/httpd/conf.modules.d/00-lua.conf /etc/httpd/conf.modules.d/00-mpm.conf /etc/httpd/conf.modules.d/00-optional.conf /etc/httpd/conf.modules.d/00-proxy.conf /etc/httpd/conf.modules.d/00-systemd.conf /etc/httpd/conf.modules.d/01-cgi.conf /etc/httpd/conf/httpd.conf /etc/httpd/conf/magic /etc/logrotate.d/httpd /etc/sysconfig/htcacheclean [root@localhost ~]# The main configuration file is /etc/httpd/conf/httpd.conf. The welcome.conf file defines the default home page for your website, until you add content. The magic file defines rules that the server can use to figure out a file\u0026rsquo;s type when the server tries to open it. The /etc/logrotate.d/httpd file defines how log files produced by Apache are rotated.\nMost Apache modules put their configuration files into the /etc/httpd/conf.d directory but some may drop their configuration files into the /etc/httpd/conf.modules.d/ directory. Any file in those directories that ends with the .conf extension is included in the main httpd.conf file and used to configure Apache.\nSetting Up a Basic Web Server # Let\u0026rsquo;s install the httpd package and some of the most commonly used additional packages using the yum module install httpd command:\n[root@localhost ~]# yum module install httpd ... ... Installed: apr-1.6.3-9.el8.x86_64 apr-util-1.6.1-6.el8.x86_64 apr-util-bdb-1.6.1-6.el8.x86_64 apr-util-openssl-1.6.1-6.el8.x86_64 centos-logos-httpd-80.5-2.el8.noarch httpd-2.4.37-21.module_el8.2.0+382+15b0afa8.x86_64 httpd-filesystem-2.4.37-21.module_el8.2.0+382+15b0afa8.noarch httpd-tools-2.4.37-21.module_el8.2.0+382+15b0afa8.x86_64 mod_http2-1.11.3-3.module_el8.2.0+307+4d18d695.x86_64 mod_ssl-1:2.4.37-21.module_el8.2.0+382+15b0afa8.x86_64 Complete! Open the main configuration file, /ect/httpd/conf/httpd.conf, and look for the DocumentRoot parameter. This parameter specifies the default location where the Apache Web Server looks for content to serve. It should be set to DocumentRoot \u0026quot;/var/www/html\u0026quot;. In the directory /var/www/html, create a file with the name index.html and the content Welcome To My Web Server!. Next, start and enable the httpd service and check if the service is up and running.\n[root@localhost ~]# echo \u0026#34;Welcome To My Webserver!\u0026#34; \u0026gt; /var/www/html/index.html [root@localhost ~]# [root@localhost ~]# systemctl enable --now httpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. [root@localhost ~]# [root@localhost ~]# systemctl status httpd ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2020-10-06 11:28:24 EST; 3s ago Docs: man:httpd.service(8) Main PID: 33293 (httpd) Status: \u0026#34;Started, listening on: port 443, port 80\u0026#34; Tasks: 213 (limit: 11323) Memory: 17.8M CGroup: /system.slice/httpd.service ├─33293 /usr/sbin/httpd -DFOREGROUND ├─33296 /usr/sbin/httpd -DFOREGROUND ├─33298 /usr/sbin/httpd -DFOREGROUND ├─33299 /usr/sbin/httpd -DFOREGROUND └─33301 /usr/sbin/httpd -DFOREGROUND Oct 06 11:28:24 localhost.localdomain systemd[1]: Starting The Apache HTTP Server... Oct 06 11:28:24 localhost.localdomain httpd[33293]: AH00558: httpd: Could not reliably determine the server\u0026#39;s fully qualified domain name, using localhost.localdomain. Set the \u0026#39;ServerName\u0026#39; directive globally to \u0026gt; Oct 06 11:28:24 localhost.localdomain systemd[1]: Started The Apache HTTP Server. Oct 06 11:28:24 localhost.localdomain httpd[33293]: Server configured, listening on: port 443, port 80 When the httpd service starts, five httpd daemon processes are launched by default to respond to requests for the web server. You can configure more or fewer daemons to be started based on settings in the main configuration file.\nWe can verify it\u0026rsquo;s working by making an http request to localhost using curl:\n[root@localhost ~]# curl http://localhost Welcome To My Webserver! Creating Apache Virtual Hosts # Apache supports the creation of separate websites within a single server. Individual sites are configured in what we refer to as virtual hosts which is just a way to have the content for multiple domain names available from the same Apache server. The content that is served to a web client is based on the (domain) name used to access the server.\nFor example, if a client got to the server by requesting the name www.example.org, he would be redirected to a virtual host container that has its ServerName parameter set to www.example.org.\nName-based virtual hosting is the most common solution where virtual hosts use different names but the same IP address. IP-based virtual hosts are less common but is required if the name of a web server must resolve to a unique IP address. This solution requires multiple IP addresses on the same machine.\nIn this section we\u0026rsquo;ll be setting up name-based virtual hosts.\nIf your Apache server is configured for virtual hosts, all sites it\u0026rsquo;s hosting should be handled by virtual hosts. If someone accesses the server via IP address or a name that is not set in a virtual host then the first virtual host is used as the default location to serve up content.\nYou can create a catch-all entry for those requests by creating a virtual host for _default:80.\nCreate a file named example.org.conf in /etc/httpd/conf.d/ using the following template:\n\u0026lt;VirtualHost *:80\u0026gt; ServerAdmin\twebmaster@example.org ServerName\texample.org ServerAlias www.example.org DocumentRoot /var/www/html/example.org/ DirectoryIndex index.php index.html index.htm \u0026lt;/VirtualHost\u0026gt; This example includes the following settings:\nThe *:80 specification indicates to what address and port this virtual host applies. If your machine has multiple IP addresses, you can replace the * with an IP. The port is optional but should always be used to prevent interference with SSL virtual hosts (which use port 443). The ServerName and ServerAlias lines tell Apache which names this virtual host should be recognized as. You can either leave out ServerAlias or specify more than one name on the same line, space separated. The DocumentRoot specifies where the content for this virtual host is stored. The DirectoryIndex directive sets the list of files to look for and serve when the web server receives a request. Create the index.html file inside the DocumentRoot with the following content: Welcome To Example.org\n[root@localhost conf.d]# mkdir /var/www/html/example.org [root@localhost conf.d]# echo \u0026#34;Welcome To Example.org\u0026#34; \u0026gt; /var/www/html/example.org/index.html Create a second virtual host with different values, e.g.:\n[root@localhost conf.d]# cat foobar.com.conf \u0026lt;VirtualHost *:80\u0026gt; ServerAdmin\twebmaster@foobar.com ServerName\tfoobar.com ServerAlias\twww.foobar.com DocumentRoot /var/www/html/foobar.com/ DirectoryIndex index.php index.html index.htm \u0026lt;/VirtualHost\u0026gt; [root@localhost conf.d]# mkdir /var/www/html/foobar.com [root@localhost conf.d]# echo \u0026#34;Welcome to Foobar.com!\u0026#34; \u0026gt; /var/www/html/foobar.com/index.html [root@localhost conf.d]# Next we want to make sure that the domains used in our virtual hosts resolve to our local machine and not to the internet. Edit your hosts file and add the domains to the line that starts with the local loopback address:\n[root@localhost conf.d]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 foobar.com www.foobar.com example.org www.example.org After we restarted the httpd service, we can test if our setup is working correctly:\n[root@localhost conf.d]# systemctl restart httpd [root@localhost conf.d]# curl http://foobar.com Welcome to Foobar.com! [root@localhost conf.d]# curl http://example.org Welcome To Example.org This covered some Apache basics which we will need for testing advanced topics like firewall configuration and SELinux.\n","date":"7 December 2021","externalUrl":null,"permalink":"/network-services-managing-apache-http/","section":"Blog","summary":"","title":"Network Services - Managing Apache HTTP","type":"posts"},{"content":" Hardening the SSH Server # SSH is a convenient and important solution to establish remote connections to servers. If your SSH server is visible directly from the internet, you can be sure that sooner or later intruders will try to connect to it, intending to do harm.\nDictionary attacks are common against an SSH server. SSH servers usually offer their services on port 22, and every Linux servers has a root account. Based on this information it\u0026rsquo;s easy for an attacker to try to log in as root by guessing the password if the password has limited complexity and no additional security measures are in place. Sooner or later the intruder will be able to connect.\nWe can protect ourselves against these kind of attacks by:\nDisabling root login. Disabling password login and using key-based authentication. Configuring a non default port for SSH to listen on. Allowing only specific users to log in on SSH. Limiting Root access # SSH servers have root login enabled by default, which is a big security concern. Disabling root login is easy: Modify the PermitRootLogin parameter in /etc/ssh/sshd_config and reload or restart the service:\n# Authentication: #LoginGraceTime 2m PermitRootLogin no #StrictModes yes #MaxAuthTries 6 #MaxSessions 10 Configuring Alternative Ports # Security problems on Linux servers start with a port scan issued by an attacker. There are 65,535 ports that can potentially be listening, and scanning all those ports takes a lot of time so most port scans focus on well known ports only. Port 22 is always among these ports.\nTo protect against port scans we can configure the SSH server to listen on another port. You can choose a completely random port, as long as the port is not already in use by another service.\n# If you want to change the port on a SELinux system, you have to tell # SELinux about this change. # semanage port -a -t ssh_port_t -p tcp #PORTNUMBER # Port 39860 #AddressFamily any #ListenAddress 0.0.0.0 #ListenAddress :: To avoid being locked out of the server after making changes to the SSH listing port, it\u0026rsquo;s a good idea to open two sessions. Use one session to apply the port change and test, use the other sessions to keep your current connection open. Active sessions will not be disconnected after restarting the SSH server (unless the restart fails), so if something is wrong with the configuration and you\u0026rsquo;re not longer able to connect you still have the second session to fix the problem.\nModifying SELinux to Allow for Port Changes # After changing the SSH port you also need to configure SELinux to allow this change. Network ports are labeled with SELinux security labels to prevent services from accessing ports they shouldn\u0026rsquo;t.\nUse the semanage port command to change the label on the target port. Before doing so, it\u0026rsquo;s a good idea to check if the port already has a label: semanage port -l, e.g. semanage port -l | grep ssh\nIf the port doesn\u0026rsquo;t have a label, use the -a option to add a label, if it does have a label use -m to modify the current security label.\nsemanage port -a -t ssh_port_t -p tcp 39860\nsemanage port -m -t ssh_port_t -p tcp 443\nLimiting User Access # The AllowUsers option takes a space separated list of usernames that will be allowed to login through SSH. If the user root still needs to be able to log in you\u0026rsquo;ll have to include it as well in the list.\nThis option does not appear anywhere in the /etc/ssh/sshd_config file by default.\nAnother interesting option is MaxAuthTries. It specifies the maximum number of authentication attempts permitted per connection.MaxAuthTries is also useful for analyzing security events, it logs failed login attempts once the number of failures reaches half this value. The higher the number of attempts, the more likely it is an intruder is trying to get in.\nSSH writes log information about failed login attempts to the AUTHPRIV syslog facility. This facility is by default configured to write information to /var/log/secure.\nOther Useful sshd Options # Apart from security-related options, there are some useful miscellaneous options you can use to streamline performance.\nSession Options # On RHEL8, GSSAPIAuthentication option is set to yes by default. This option is only useful in an environment where Kerberos authentication is used. Having this feature on slows down the authentication procedure.\nThe UseDNS option is also enabled by default and instructs the SSH server to lookup the remote hostname and check with DNS that the hostname maps back to the same IP address (Reverse DNS Lookup). Although this option has some security benefits, it also involves a significant performance penalty. Set this to no if client connections are slow.\nTo give an example on Reverse DNS lookups, assume you\u0026rsquo;re connecting from a client with the 8.8.8.8 ip address. The SSH server will lookup the PTR record for the 8.8.8.8.in-addr.arpa domain which would result in dns.google. In turn, this result resolves back to 8.8.8.8. The reverse DNS database of the Internet is rooted in the .arpa top-level domain.\n$ dig -x 8.8.8.8 ;; ANSWER SECTION: 8.8.8.8.in-addr.arpa.\t76082\tIN\tPTR\tdns.google. $ dig dns.google ;; ANSWER SECTION: dns.google.\t824\tIN\tA\t8.8.4.4 dns.google.\t824\tIN\tA\t8.8.8.8 The MaxSessions option specifies the maximum number of sessions that can be opened from one IP address simultaneously. You might need to increase this option beyond the default value of 10.\nConnection Keepalive Options # The TCPKeepAlive option is used to monitor whether the client is still available. This option is by default enabled and sends a keepalive probe packet with the ACK flag to the client after a certain amount of time. If a reply is received, the SSH server can assume that the connection is still up and running.\nThe ClientAliveInterval option sets an interval in seconds after which the server sends a packet to the client if no activity has been detected. The ClientAliveCountMax parameter specifies how many of these should be sent. So if the ClientAliveInterval is set to 30 and the ClientAliveCountMax to 10, inactive connections are kept alive for about 5 minutes.\nThe equivalent client side options are ServerAliveInterval and ServerAliveCountMax, useful if you cannot change the configuration of the SSH server.\nConfiguring Key-Based Authentication with Passphrases # By default, password authentication is allowed on RHEL 8 SSH servers. You can disable password authentication and allow public/private key-based authentication only by setting the PasswordAuthentication option to no.\nWhen using key-based authentication you can set a passphrase which makes the key pair stronger. In case an intruder has access to the private key he would also need to know the passphrase before being able to use the key.\nWithout further configuration the use of passphrases would mean that users have to enter the passphrase every time before a connection can be created, which is inconvenient. To work around this we can cache the passphrase for a session:\nExecute the ssh-agent /bin/bash command to start the agent for the current (Bash) shell. Execute ssh-add to add the passphrase for the current user\u0026rsquo;s private key. The key is now cached. Connect to the remote server, you\u0026rsquo;ll notice you do not need to enter the passphrase. Copying and synchronizing files securely over SSH # scp is a program for copying files securely between computers using the SSH protocol.\nThe basic usage is as follows:\nTo copy a local file to a remote host: scp localfile remote_host:remote_path To copy a remote file to a local path: scp remote_host:remote_file localpath To copy entire directory trees, add the -r option: scp -r remote_host:path/directory . Rsync, which stands for “remote sync”, is a remote and local file synchronization tool. It uses an algorithm that minimizes the amount of data copied by only moving the portions of files that have changed. The basic syntax is similar to that of scp: rsync source destination.\nrsync -anvzP --progress remote_host:/path/to/directory/ /some/local/path The -a option is a combination flag, it stands for \u0026ldquo;archive\u0026rdquo; and syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions. You could use -r to only sync recursively instead. The -n flag is the same as the --dry-run option and allows you to check results before actually running the synchronization. You need the -v flag (verbose) to get the appropriate output to verify.\nThe -z option can reduce network transfer by adding compression.\nThe -P flag combines the --progress and --partial options, it gives you a progress bar and allows you to resume interrupted transfers.\nFinally, you can use the -A flag to preserve Access Control Lists, and the -X flag to preserve SELinux context labels.\nNotice the traling slash / at the end of the first argument in the example command. This is necessary to include the contents of the source path. Without the trailing slash, directory would be created inside /some/local/path.\n","date":"14 August 2021","externalUrl":null,"permalink":"/network-services-configuring-ssh/","section":"Blog","summary":"","title":"Network Services - Configuring SSH","type":"posts"},{"content":"","date":"14 August 2021","externalUrl":null,"permalink":"/tags/ssh/","section":"Tags","summary":"","title":"SSH","type":"tags"},{"content":"","date":"24 July 2021","externalUrl":null,"permalink":"/tags/bash/","section":"Tags","summary":"","title":"Bash","type":"tags"},{"content":" Core Elements # A shell script is a list of sequentially executed commands with optional scripting logic to allow code to be executed under specific conditions only. Starting a script from the parent shell opens a subshell from where the commands in the script are executed. These commands can be interpreted in different ways, to make it clear how they should be interpreted the shebang is used on the first line of the script: #!/bin/bash, which would call and execute the script in a bash subshell.\nThe below script asks you for a path and stores the path in the DIR variable, then changes directory to the DIR value and prints the current working directory.\n#!/bin/bash # MyComment echo Provide a path to a directory: read DIR cd $DIR pwd exit 0 When you execute this script, notice how your current working directory hasn\u0026rsquo;t changed after the script has executed. This is because the script executes in a subshell of the parent shell from where you invoked the script.\nAt the end of the above script an exit 0 statement is included. An exit statement tells the parent script whether the scipt was successful, a 0 means it was successfull, while anything else means a problem was encountered.\nA script needs to be executable. The most common way to make a script executable is by applying the execute permission to it. The script can also be executed as an argument to the bash command, e.g. bash myscript.sh.\nYou can store a script anywhere you like, but if it\u0026rsquo;s stored outide of the $PATH you need to execute it with a ./ in front: ./myscript.sh.\nVariables and Input # Scripts typically aren\u0026rsquo;t a list of sequential commands, they can work with variables and input to be more flexible.\nPositional Parameters # When starting a script, an argument can be used. Arguments are anything you put behind the command while starting the script, e.g. useradd lisa where the command is useradd and the argument is lisa. In a script, the first variable is referred to as $1, the second as $2 and so on.\n#!/bin/bash # Run this script with a few arguments echo The first argument is $1 echo The 2nd argument is $2 echo The 3rd argument is $3 Run the above script with a few arguments, and it will make sense: ./script 1 2 3 4 You\u0026rsquo;ll notice the 4th argument, being 4 isn\u0026rsquo;t echoed. We can work around that by making the script more flexible using a conditional for loop instead of echoeing each argument one after the other:\n#!/bin/bash # Run this script with a few arguments echo You have entered $# arguments. for i in \u0026#34;$@\u0026#34; do echo $i done exit 0 $# is a counter that shows how many arguments were used when starting the script.\n$@ refers to all arguments used when starting the script. In the above script, the condition is for i in \u0026quot;$@\u0026quot;, which means \u0026ldquo;for each argument in the list of arguments\u0026rdquo;. I\u0026rsquo;ll cover more on for loops later, but what this script basically does is loop through the list of arguments ($@) and echo each one (do echo $i).\nVariables # Variables are labels that refer to a specific location in memory which contains a specific value. They can be defined statically or dynamically. Variables are defined by using the = sign directly after the uppercase name, followed by the value. You should never use spaces when defining variables:\nMYVAR=value, this would be a statically defined variable.\nThere are two solutions for defining variables dynamically:\nUsing read in the script to ask the user for input. IT stops the script so input can be processed and stored in a variable: [joeri@Ryzen7 ~]$ read NAME joeri [joeri@Ryzen7 ~]$ echo $NAME joeri Using command substitution where you assign the result of a specific command to a variable. For example: TODAY=$(date +%d-%m-%y).\nYou enclose the command whose result you want to use between parentheses and preceed that with a $ sign. [joeri@Ryzen7 ~]$ TODAY=$(date +%d-%m-%y) [joeri@Ryzen7 ~]$ echo $TODAY 31-10-20 Conditional Loops # Conditional loops are executed only if a certain condition is true. I\u0026rsquo;ll cover the most often used conditional loops in this section.\nif \u0026hellip; then \u0026hellip; else # This construction is common to evaluate specific conditions and are often used together with the test command. Have a look at the man page of test for a complete overview of all the functionality.\nLet\u0026rsquo;s look at an example:\n#!/bin/bash # MyComment if [ -z $1 ] then echo No value provided fi The -z test command checks if the length of a string is zero (man test). If that is true, then \u0026ldquo;No value provided\u0026rdquo; will be echoed to the screen. The above script will only provide output if you run it without any argument.\nBelow is another example using multiple test commands:\n#!/bin/bash # Run this script with one argument. # Find out if the argument is a file or a directory if [ -f $1 ] then echo \u0026#34;$1 is a file\u0026#34; elif [ -d $1 ] then echo \u0026#34;$1 is a directory\u0026#34; else echo \u0026#34;Not sure what $1 is....\u0026#34; fi exit 0 || and \u0026amp;\u0026amp; # Instead of writing full if ... then statements we can use logical operators. || is a logical OR and will execute the second part of the statement only if the first part is not true. \u0026amp;\u0026amp; is a logical AND, and will execute the second part of the statement only if the first part is true. \u0026ldquo;true\u0026rdquo; is the state where a command exits with a 0.\n[ -z $1 ] \u0026amp;\u0026amp; echo no argument provided ping -c 1 192.168.1.256 || echo node does not exist For \u0026hellip; do \u0026hellip; done # The for conditional loop provides a solution for processing ranges of data. It always starts with for followed by the condition, then do followed by the commands to be executed when the condition is true, and finally closed with done.\nIn the below example the COUNTER variable is initialized with a value of 10, if the value is greater than or equal to 0 we substract 1. As long as this condition is true we then echo the value of COUNTER:\n#!/bin/bash # for (( COUNTER=10; COUNTER\u0026gt;=0; COUNTER--)) do echo $COUNTER done exit 0 We can also define a range by specifying the first number followed by two dots and closing with the last number in the range:\n[joeri@Ryzen7 ~]$ for i in {85..90}; do ping -c 1 192.168.100.$i \u0026gt;/dev/null \u0026amp;\u0026amp; echo 192.168.100.$i is UP; done 192.168.100.88 is UP With for i in each of the numbers in the range is assigned to the variable i. For each of those values the ping -c 1 command is executed, and output is redirected to /dev/null since we don\u0026rsquo;t need it. Based on the exit status of the ping command, exit 0 or true, the part behind the logical operator \u0026amp;\u0026amp; is executed.\nWhile and until # The while statement is useful if you want to do something as long as a condition is true. Its counterpart is until which keeps the iteration open as long as the condition is false, or until the condition is true.\nThe below script initializes the COUNTER value with a value of 0 and while the value is less than 11 we echo the value and increase the value with 1:\n#!/bin/bash # COUNTER=0 while [ $COUNTER -lt 11 ]; do echo The counter is $COUNTER (( COUNTER=COUNTER+1 )) done Below we echo the value of COUNTER and increase its value with 1 until the value is equal to 11. At that point we break out of the loop:\n#!/bin/bash # COUNTER=0 until [ $COUNTER = 11 ]; do echo The counter is $COUNTER (( COUNTER=COUNTER+1 )) done Case # The case statement is used to evaluate a number of expected values, you define very specific argument that you expect followed by the command that needs to be executed if that argument was used.\nThe generic syntax is case item-to-evaluate in, followed by a list of all possible values that need to be evaluated. Each item is closed with a ). Then follows a list of commands that are executed if the specific argument was used, the commands are closed with a double semicolon, ;;.\nThe evaluations in case are performed in order. Then the first match is made, the case statement will not evaluate anything else. Whitin the evaluaten, wildcard-like patterns can be used. For example *), which is a \u0026ldquo;catchall\u0026rdquo; statement.\n#!/bin/bash echo -n \u0026#34;Enter the name of a country: \u0026#34; read COUNTRY echo -n \u0026#34;The official language of $COUNTRY is \u0026#34; case $COUNTRY in Lithuania) echo -n \u0026#34;Lithuanian\u0026#34; ;; Romania | Moldova) echo -n \u0026#34;Romanian\u0026#34; ;; Italy | \u0026#34;San Marino\u0026#34; | Switzerland | \u0026#34;Vatican City\u0026#34;) echo -n \u0026#34;Italian\u0026#34; ;; *) echo -n \u0026#34;unknown\u0026#34; ;; esac Script debugging # If a script does not do what you expect it to do, try starting it as an argument to the bash -x command. This will show you line by line what the script is trying to do and will show specific errors if it does not work.\n[joeri@Ryzen7 ~]$ bash -x lang.sh + echo -n \u0026#39;Enter the name of a country: \u0026#39; Enter the name of a country: + read COUNTRY Germany + echo -n \u0026#39;The official language of Germany is \u0026#39; The official language of Germany is + case $COUNTRY in + echo -n unknown unknown ","date":"24 July 2021","externalUrl":null,"permalink":"/introduction-to-bash-shell-scripting/","section":"Blog","summary":"","title":"Introduction to Bash Shell Scripting","type":"posts"},{"content":"","date":"24 July 2021","externalUrl":null,"permalink":"/tags/scripting/","section":"Tags","summary":"","title":"Scripting","type":"tags"},{"content":"","date":"24 July 2021","externalUrl":null,"permalink":"/tags/shell/","section":"Tags","summary":"","title":"Shell","type":"tags"},{"content":"","date":"3 July 2021","externalUrl":null,"permalink":"/tags/boot-arguments/","section":"Tags","summary":"","title":"Boot Arguments","type":"tags"},{"content":"","date":"3 July 2021","externalUrl":null,"permalink":"/tags/file-system-issues/","section":"Tags","summary":"","title":"File System Issues","type":"tags"},{"content":"","date":"3 July 2021","externalUrl":null,"permalink":"/tags/initramfs/","section":"Tags","summary":"","title":"Initramfs","type":"tags"},{"content":"","date":"3 July 2021","externalUrl":null,"permalink":"/tags/rescue-disk/","section":"Tags","summary":"","title":"Rescue Disk","type":"tags"},{"content":"","date":"3 July 2021","externalUrl":null,"permalink":"/tags/root-password/","section":"Tags","summary":"","title":"Root Password","type":"tags"},{"content":"","date":"3 July 2021","externalUrl":null,"permalink":"/tags/troubleshooting/","section":"Tags","summary":"","title":"Troubleshooting","type":"tags"},{"content":" The RHEL8 Boot Procedure # In order to fix boot issues we need to be able to judge in which phase of the boot procedure the issue occurs so we can apply appropriate means to fix it. The following steps summarize the boot procedure:\nPOST - The machine is powered on, the Power-On-Self-Test executes and hardware required to start the system is initialized. Boot device selection - From UEFI or BIOS, a bootable device is located. Loading the boot loader - From the bootable device a boot loader is located. Loading the kernel - The kernel is loaded together with the initramfs. The initramfs contains kernel modules required to boot as well as initials scripts to proceed to the next stage of booting. Starting /sbin/init - The first process is loaded, /sbin/init, which is a symlink to Systemd. The udev daemon is loaded to take care of further hardware initialization. This all happens from initramfs. Process initrd.target - The Systemd process executes all units from the initrd.target, preparing a minimal operating environment from where the root file system on disk is mounted onto the /sysroot directory. Switch to root file system - The system switches to the root file system on disk and loads the Systemd process from disk. Running the default target - Systemd looks for the default target to execute and runs all of its units. The below table summarizes where a specific phase is configured and what you can do to troubleshoot if something goes wrong.\nPhase Configuration Fix POST Hardware Configuration, BIOS, UEFI Replace Hardware Boot Device BIOS/UEFI configuration or boot menu Replace hardware or use rescue system Boot Loader grub2-install and edits to /etc/defaults/grub GRUB Boot menu, edits to /etc/defaults/grub followed by grub2-mkconfig Kernel Edits to GRUB config and /etc/dracut.conf GRUB Boot menu, edits to /etc/defaults/grub followed by grub2-mkconfig /sbin/init Compiled into initramfs init= kernel boot argument, rd.break kernel boot argument, recreate initramfs initrd.target Compiled into initramfs recreate initramfs Root file system Edits to /etc/fstab Edits to /etc/fstab Default Target systemctl set-default Start rescue.target as a kernel boot argument Passing Kernel Boot Arguments # The GRUB boot prompt offers a way to stop the boot procedure and pass specific options to the kernel. When you see the GRUB2 menu, type e to enter a mode where you can edit commands and scroll down to the section that begins with linux ($root)/vmlinuz. This line tells GRUB how to start a kernel and looks similar to this:\nlinux ($root)/vmlinuz-4.18.0-193.19.1.el8_2.x86_64 root=/dev/mapper/cl-root ro crash kernel-auto resume=/dev/mapper/cl-swap rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet Additional boot arguments need to be added to the end of this line.\nThe rhgb and quiet boot options hide boot messages, we can remove these in order to see what\u0026rsquo;s happening when we boot the machine. Once you made the necessary changes, press CTRL+X to start the kernel. Note that this change is not persistent, to make them persistent we must modify the content of /etc/default/grub and use grub2-mkconfig -o /boot/grub2/grub.cf to apply the change.\nStarting a Troubleshooting Target # In the GRUB boot prompt we can use several options to allow us to fix our issue:\nrd.break - Stops the boot procedure in the initramfs phase. This option is useful if you don\u0026rsquo;t have the root password. init=/sbin/bash - A shell will be started immediately after loading the kernel and initrd.target. systemd.unit=emergency.target - Enters a mode that loads the bare minimum of required Systemd units, it requires a root password. systemd.unit=rescue.target - Starts more Systemd units to bring up a more complete operational mode. Using a Rescue Disk # The default rescue image for RHEL is on the installation disk. When booting from the installation disk you\u0026rsquo;ll see a Troubleshooting menu item which presents you with the following options:\nInstall RHEL in Basic Graphics Mode - This option reinstalls the machine. You should not use it unless a normal installation does not work and you need basic graphics mode. Rescue a RHEL System - This options prompts you to press Enter to start the installation, but only loads a rescue system. It does not overwrite the current configuration. The Rescue System will try to find an installed Linux system and mount it on /mnt/sysimage. If a valid installation was found and mounted you can press Enter twice to access the rescue shell. At this point we can switch to the root file system on disk to access all tools we need to repair the system: chroot /mnt/sysimage Run a Memory Test - If you encounter memory errors this tool allows you to mark bad memory chips so you can boot your machine normally. Boot from Local Drive - If you cannot boot from GRUB on your usual boot device try this option. It offers a boot loader that will try to load the OS from your hard disk. Reinstalling Grub Using a Rescue Disk # One of the most common reasons to start a rescue disk is if the GRUB2 boot loader breaks. Once you have access to your machine using the rescue disk, reinstalling GRUB2 is a two step process:\nMake sure you switch to the root file system on disk: chroot /mnt/sysroot Use grub2-install followed by the name of the device on which you want to reinstall GRUB2, i.e. grub2-install /dev/sda Recreating Initramfs Using a Rescue Disk # You know there is a problem with initramfs when you never see the root file system getting mounted on the root directory and don\u0026rsquo;t see any Systemd unit files being started when analyzing the boot procedure.\nTo repair the initramfs image after booting into the rescue environment you can use the dracut command. dracut \u0026ndash;force overwrites the existing initramfs and creates a new initramfs image for the currently loaded kernel. There is also the /etc/dracut.conf configuration file you can use to include specific options while re-creating initramfs. The dracut configuration itself is dispersed over several locations:\n/usr/lib/dracut/dracut.conf.d/ - Contains the system default configuration files /etc/dracut.conf.d/ - Contains custom dracut configuration files /etc/dracut.conf - The master configuration file Recovering from File System Issues # When there is a misconfiguration in the file system mounts the boot procedure may end with the \u0026ldquo;Give root password for maintenance\u0026rdquo; message. If a device does not exist or there\u0026rsquo;s an error in the UUID, for example, Systemd waits to see if the device comes back online by itself. When that doesn\u0026rsquo;t happen, the \u0026ldquo;Give root password for maintenance\u0026rdquo; message appears.\nAfter entering the root password, issue the journalctl -xb command to see if relevant messages providing information about what is wrong are written to the journal. If the problem is indeed file system oriented we need to make sure the root file system is mountend with read/write rights, analyze what\u0026rsquo;s wrong in /etc/fstab and fix that: mount -o remount,rw /\nResetting the Root Password # When the root password is lost, the only way to reset it is to boot into minimal mode which allows you to login without using a password:\nPass the rd.break boot argument to the kernel Boot the system The boot procedure stops after loading initramfs and before mounting the root file system. Re-mount the root file system on disk to get read/write access to the system image: mount -o remount,rw /sysroot Make the contents of the /sysroot directory the new root directory: chroot /sysroot Use the passwd command to set the new password. Load the SELinux policy: load_policy -i Set the correct SELinux contect type to /etc/shadow: chcon -t shadow_t /etc/shadow Reboot by issuing the exit command twice. Use the new root password at the next boot. An alternative to applying the SELinux context to /etc/shadow is to create the /.autorelabel file which forces SELinux to restore labels set on the entire file system the next time the system is booted.\n","date":"3 July 2021","externalUrl":null,"permalink":"/troubleshooting-boot-issues/","section":"Blog","summary":"","title":"Troubleshooting Boot Issues","type":"posts"},{"content":"","date":"11 June 2021","externalUrl":null,"permalink":"/tags/boot/","section":"Tags","summary":"","title":"Boot","type":"tags"},{"content":"","date":"11 June 2021","externalUrl":null,"permalink":"/tags/grub2/","section":"Tags","summary":"","title":"GRUB2","type":"tags"},{"content":" Managing Systemd Targets # A Systemd target is a group of units belonging together, some of these targets can be used to define the state a system should boot in. These targets can be isolated and have the AllowIsolate property in their [Unit] section.\nFour targets can be used to boot into:\nemergency.target : A minimal number of units are started. rescue.target : A fully operation Linux system without nonessential services. multi-user.target : The default target commonly used on servers, starts everything needed for full system functionality. graphical.target : Starts all units needed for full system functionality as well as a graphical interface. A target configuration consists of two parts, the target unit file and the \u0026ldquo;wants\u0026rdquo; directory that contains references to all unit files that need to be loaded when entering that specific target. They can also have other targets as dependencies, specified in the target unit file.\n[root@server1 ~]# systemctl cat multi-user.target # /usr/lib/systemd/system/multi-user.target # SPDX-License-Identifier: LGPL-2.1+ # # This file is part of systemd. # # systemd is free software; you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation; either version 2.1 of the License, or # (at your option) any later version. [Unit] Description=Multi-User System Documentation=man:systemd.special(7) Requires=basic.target Conflicts=rescue.service rescue.target After=basic.target rescue.service rescue.target AllowIsolate=yes The target unit doesn\u0026rsquo;t contain much, it defines what it requires and which services and targets it can\u0026rsquo;t coexist with. The After statement in the [Unit] sections also defines load ordering. It does not contain any information about units that it \u0026ldquo;wants\u0026rdquo;.\nUnderstanding Wants # Wants define which units should start when booting or starting a specific target. Wants are created when enabling units using systemd enable, this happens by creating a symbolic link in the /etc/systemd/system directory. This directory contains a subdirectory for every target, which in turn contains \u0026ldquo;wants\u0026rdquo; as symbolic links to specific services that should be started:\n[root@server1 ~]# ls -l /etc/systemd/system/multi-user.target.wants/ total 0 lrwxrwxrwx. 1 root root 35 Sep 26 17:46 atd.service -\u0026gt; /usr/lib/systemd/system/atd.service lrwxrwxrwx. 1 root root 38 Sep 26 17:44 auditd.service -\u0026gt; /usr/lib/systemd/system/auditd.service lrwxrwxrwx. 1 root root 44 Sep 26 17:46 avahi-daemon.service -\u0026gt; /usr/lib/systemd/system/avahi-daemon.service lrwxrwxrwx. 1 root root 39 Sep 26 17:45 chronyd.service -\u0026gt; /usr/lib/systemd/system/chronyd.service lrwxrwxrwx. 1 root root 37 Sep 26 17:44 crond.service -\u0026gt; /usr/lib/systemd/system/crond.service ... The [Install] section in a service unit file specifies the target it is \u0026ldquo;wanted\u0026rdquo; by. Enabling the service creates a symbolic link in that targets\u0026rsquo; \u0026ldquo;wants\u0026rdquo; directory, making sure it starts when that target is booted into or started.\n[root@server1 ~]# systemctl cat httpd.service ... [Install] WantedBy=multi-user.target [root@server1 ~]# systemctl enable httpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. Isolating Targets # To get a list of all targets that are currently loaded, we can use the systemctl --type=target command. This shows all currently active targets. The systemctl --type=target --all command also shows inactivate targets.\n[root@server1 ~]# systemctl --type=target UNIT LOAD ACTIVE SUB DESCRIPTION basic.target loaded active active Basic System cryptsetup.target loaded active active Local Encrypted Volumes getty.target loaded active active Login Prompts graphical.target loaded active active Graphical Interface local-fs-pre.target loaded active active Local File Systems (Pre) local-fs.target loaded active active Local File Systems multi-user.target loaded active active Multi-User System network-online.target loaded active active Network is Online network.target loaded active active Network nfs-client.target loaded active active NFS client services nss-user-lookup.target loaded active active User and Group Name Lookups paths.target loaded active active Paths remote-fs-pre.target loaded active active Remote File Systems (Pre) remote-fs.target loaded active active Remote File Systems rpc_pipefs.target loaded active active rpc_pipefs.target rpcbind.target loaded active active RPC Port Mapper slices.target loaded active active Slices sockets.target loaded active active Sockets sound.target loaded active active Sound Card sshd-keygen.target loaded active active sshd-keygen.target swap.target loaded active active Swap sysinit.target loaded active active System Initialization timers.target loaded active active Timers LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 23 loaded units listed. Pass --all to see loaded but inactive units, too. To show all installed unit files use \u0026#39;systemctl list-unit-files\u0026#39;. Some of these targets can be isolated, they can be started to define the state of the machine and these are also the targets that can be set as the default target. They roughly correspond to the following System V runlevels:\nTarget Runlevel poweroff.target runlevel 0 rescue.target runlevel 1 multi-user.target runlevel 3 graphical.target runlevel 5 reboot.target runlevel 6 As mentioned earlier, targets that can be isolated have the AllowIsolate property in their [Unit] section:\n[root@server1 system]# grep Isolate *.target anaconda.target:AllowIsolate=yes ctrl-alt-del.target:AllowIsolate=yes default.target:AllowIsolate=yes emergency.target:AllowIsolate=yes exit.target:AllowIsolate=yes graphical.target:AllowIsolate=yes halt.target:AllowIsolate=yes initrd-switch-root.target:AllowIsolate=yes initrd.target:AllowIsolate=yes kexec.target:AllowIsolate=yes multi-user.target:AllowIsolate=yes poweroff.target:AllowIsolate=yes reboot.target:AllowIsolate=yes rescue.target:AllowIsolate=yes runlevel0.target:AllowIsolate=yes runlevel1.target:AllowIsolate=yes runlevel2.target:AllowIsolate=yes runlevel3.target:AllowIsolate=yes runlevel4.target:AllowIsolate=yes runlevel5.target:AllowIsolate=yes runlevel6.target:AllowIsolate=yes system-update.target:AllowIsolate=yes To switch the current state of your machine to either one of these targets, use the systemctl isolate command: systemctl isolate rescue.target systemctl isolate reboot.target\nWe can set a default ttarget using the systemctl set-default command, or check the current default target using the systemctl get-default command. Notice how the existing symlink is removed and a new one is created for default.target:\n[root@server1 system]# systemctl get-default graphical.target [root@server1 system]# systemctl set-default multi-user.target Removed /etc/systemd/system/default.target. Created symlink /etc/systemd/system/default.target → /usr/lib/systemd/system/multi-user.target. Working with GRUB2 # The GRUB2 bootloader makes sure we can boot Linux, it\u0026rsquo;s installed in the boot sector of the hard drive and loads a Linux kernel and initramfs. The initramfs contains a mini file system, mounted during boot, from where kernel modules load that are needed during the rest of the boot process, e.g. LVM modules.\nWe apply changes to GRUB2 by editing the /etc/default/grub file and we pass boot arguments to the kernel by editing the GRUB_CMDLINE_LINUX line:\n[root@server1 system]# cat /etc/default/grub GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR=\u0026#34;$(sed \u0026#39;s, release .*$,,g\u0026#39; /etc/system-release)\u0026#34; GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT=\u0026#34;console\u0026#34; GRUB_CMDLINE_LINUX=\u0026#34;crashkernel=auto resume=/dev/mapper/cl-swap rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet\u0026#34; GRUB_DISABLE_RECOVERY=\u0026#34;true\u0026#34; GRUB_ENABLE_BLSCFG=true The GRUB_TIMEOUT parameter defines how long GRUB2 waits before proceeding with the boot procedure. During this time you can press e to make changes to the configuration, just as you would by editing the /etc/default/grub file.\nRemoving the rhgb and quiet boot options would allow you to see the output of the boot procedure on screen.\nAfter making changes to /etc/default/grub the relevant GRUB file on the /boot partition needs to be regenerated. On a BIOS system this file is located in /boot/grub2/grub.cfg, while on a UEFI system the file is located in /boot/efi/EFI/redhat/grub.cfg. To regenerate these files, we issue the grub2-mkconfig command and redirect its output to either one of these files: grub2-mkconfig -o /boot/grub2/grub.cfg grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg\n","date":"11 June 2021","externalUrl":null,"permalink":"/managing-systemd-targets-working-with-grub2/","section":"Blog","summary":"","title":"Managing Systemd Targets and Working with GRUB2","type":"posts"},{"content":"","date":"11 June 2021","externalUrl":null,"permalink":"/tags/systemd-targets/","section":"Tags","summary":"","title":"Systemd Targets","type":"tags"},{"content":" The Role of the Linux Kernel # The Linux kernel is the layer between the user who works with Linux from a shell environment and the available hardware. It manages the I/O instructions received from software and translates it to CPU instructions. The kernel also handles essential operating system tasks like the scheduler to make sure that any processes started on the OS are handled by the CPU.\nOS tasks that are handled by the kernel are implemented by using different kernel threads. You can easily indentify them with a command like ps aux, the kernel threads are listed between square brackets:\n[root@server1 ~]# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.5 180072 10364 ? Ss 09:50 0:01 /usr/lib/syst root 2 0.0 0.0 0 0 ? S 09:50 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? I\u0026lt; 09:50 0:00 [rcu_gp] root 4 0.0 0.0 0 0 ? I\u0026lt; 09:50 0:00 [rcu_par_gp] root 6 0.0 0.0 0 0 ? I\u0026lt; 09:50 0:00 [kworker/0:0H root 8 0.0 0.0 0 0 ? I\u0026lt; 09:50 0:00 [mm_percpu_wq root 9 0.0 0.0 0 0 ? S 09:50 0:00 [ksoftirqd/0] root 10 0.0 0.0 0 0 ? I 09:50 0:00 [rcu_sched] The kernel also handles hardware initialization, making sure hardware can be used. To do so, drivers must be loaded and since the kernel is modular these drivers are loaded as kernel modules.\nHardware manufacturers do not always provide open source drivers, in this case the alternative would be to use closed source drivers. This is not always ideal, a badly functioning driver can crash the entire kernel. If this happens on an open source driver, the Linux community would jump in to debug and fix the problem which cannot be done on a closed source driver. A closed source or proprietary driver may however provide additional functionality not available in the open source equivalent. A kernel that is using closed source drivers is known as a tainted kernel.\nAnalyzing What the Kernel is Doing # A few different tools are provided by the Linux operating system to help check what the kernel is doing:\ndmesg The /proc pseudo file system The uname and hostnamectl utility When you require detailed information about kernel activity, you can use the dmesg command. This prints the content of the kernel ring buffer, an area of memory where the kernel keeps the recent log messages. Each entry in the output starts with a time indicator that shows the specific second the event was logged, relative to the start of the kernel.\nAn alternative to dmesg is journalctl - -dmesg or journalctl -k. These commands show a clock time indicator.\n[root@server1 ~]# dmesg | head [ 0.000000] Linux version 4.18.0-193.19.1.el8_2.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP Mon Sep 14 14:37:00 UTC 2020 [ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/vmlinuz-4.18.0-193.19.1.el8_2.x86_64 root=/dev/mapper/cl-root ro crashkernel=auto resume=/dev/mapper/cl-swap rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet [ 0.000000] x86/fpu: x87 FPU will use FXSAVE [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007ffdcfff] usable [ 0.000000] BIOS-e820: [mem 0x000000007ffdd000-0x000000007fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved [root@server1 ~]# journalctl -k | head -- Logs begin at Fri 2020-10-02 09:50:11 +04, end at Fri 2020-10-02 10:53:20 +04. -- Oct 02 09:50:11 server1.example.local kernel: Linux version 4.18.0-193.19.1.el8_2.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP Mon Sep 14 14:37:00 UTC 2020 Oct 02 09:50:11 server1.example.local kernel: Command line: BOOT_IMAGE=(hd0,msdos1)/vmlinuz-4.18.0-193.19.1.el8_2.x86_64 root=/dev/mapper/cl-root ro crashkernel=auto resume=/dev/mapper/cl-swap rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet Oct 02 09:50:11 server1.example.local kernel: x86/fpu: x87 FPU will use FXSAVE Oct 02 09:50:11 server1.example.local kernel: BIOS-provided physical RAM map: Oct 02 09:50:11 server1.example.local kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable Oct 02 09:50:11 server1.example.local kernel: BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved Oct 02 09:50:11 server1.example.local kernel: BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved Oct 02 09:50:11 server1.example.local kernel: BIOS-e820: [mem 0x0000000000100000-0x000000007ffdcfff] usable Oct 02 09:50:11 server1.example.local kernel: BIOS-e820: [mem 0x000000007ffdd000-0x000000007fffffff] reserved Many of the performance related commands or tools we use grab their information from the /proc file system. It contains detailed status information about what is happening on the machine. The /proc directory contains Process ID subdirectories which contain information about the particular process. The directory also contains status files, i.e. /proc/partitions or /proc/meminfo:\n[root@server1 ~]# cat /proc/1146/status Name:\tkvdo0:cpuQ0 Umask:\t0000 State:\tS (sleeping) Tgid:\t1146 Ngid:\t0 Pid:\t1146 PPid:\t2 ... [root@server1 ~]# cat /proc/partitions major minor #blocks name 11 0 8038400 sr0 8 64 5242880 sde 8 48 5242880 sdd 8 0 26214400 sda 8 1 1048576 sda1 8 2 25164800 sda2 8 32 5242880 sdc 8 16 5242880 sdb 8 17 102400 sdb1 8 18 921600 sdb2 8 19 2097152 sdb3 8 20 2120687 sdb4 ... [root@server1 ~]# cat /proc/meminfo MemTotal: 1870616 kB MemFree: 87464 kB MemAvailable: 184724 kB ... We can change kernel performance parameters during run time by writing values to the /proc/sys pseudo file system. You can apply the changes permanently by writing the parameters to /etc/sysctl.conf. To see what parameters are currently in use, issue the systctl -a command.\n[root@server1 ~]# sysctl -a | grep ip_forward net.ipv4.ip_forward = 0 [root@server1 ~]# echo \u0026#34;1\u0026#34; \u0026gt; /proc/sys/net/ipv4/ip_forward [root@server1 ~]# sysctl -a | grep ip_forward net.ipv4.ip_forward = 1 [root@server1 ~]# echo \u0026#34;net.ipv4.ip_forward = 1\u0026#34; \u0026gt;\u0026gt; /etc/sysctl.conf [root@server1 ~]# reboot [root@server1 ~]# sysctl -a | grep ip_forward net.ipv4.ip_forward = 1 Another useful command would be uname and hostnamectl, it gives different kinds of information about the OS:\n[root@server1 ~]# uname -a Linux server1.example.local 4.18.0-193.19.1.el8_2.x86_64 #1 SMP Mon Sep 14 14:37:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux [root@server1 ~]# uname -r 4.18.0-193.19.1.el8_2.x86_64 [root@server1 ~]# hostnamectl status Static hostname: server1.example.local Icon name: computer-vm Chassis: vm Machine ID: e40db12b26ec4e9bb6a6f295f6d4d83e Boot ID: 5441210211394c5098724e9b89426cb2 Virtualization: kvm Operating System: CentOS Linux 8 (Core) CPE OS Name: cpe:/o:centos:centos:8 Kernel: Linux 4.18.0-193.19.1.el8_2.x86_64 Architecture: x86-64 Lastly, you can cat the distribution release verion:\n[root@server1 ~]# cat /etc/redhat-release CentOS Linux release 8.2.2004 (Core) Working with Kernel Modules # Since the release of Linux kernel 2.0 kernels are no longer compiled but modular. A modular kernel consists of a relatively small kernel core and provides driver support through modules that are loaded when they are required. Modules implement specific kernel functionality, they are not limited to loading hardware drivers alone. For example, file system support is also loaded as kernel modules.\nUnderstanding Hardware Initialization # The loading of drivers is an automated process:\nThe kernel probes available hardware during boot. When a hardware component is detected, the systemd-udevd process loads the appropriate driver and makes the device available. systemd-udevd reads the rules in /usr/lib/udev/rules.d/. These are system-provided rules that should not be modified. systemd-udevd reads custom rules from the /etc/udev/rules.d directory, if available. Required kernel modules have been loaded and the status of associated hardware is written to the sysfs file system on /sys. This pseudo file system tracks hardware-related settings. The systemd-udevd process continuously monitors for plugging and unplugging of hardware devices. You can see this in action when plugging/unplugging an usb or other block device while the udevadm monitor command is running:\n[root@server1 ~]# udevadm monitor monitor will print the received events for: UDEV - the event which udev sends out after rule processing KERNEL - the kernel uevent KERNEL[7080.543250] change /devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sr0 (block) UDEV [7080.558849] change /devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sr0 (block) KERNEL[7080.578292] change /devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sr0 (block) UDEV [7080.746283] change /devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sr0 (block) Managing Kernel Modules # Although loading of drivers happens automatically when they are required, there might be occasions where you need to manually load the appropriate kernel module.\nTo list all currently used kernel modules we use the lsmod command:\n[root@server1 ~]# lsmod | head Module Size Used by binfmt_misc 20480 1 nls_utf8 16384 1 isofs 45056 1 fuse 131072 3 uinput 20480 1 xt_CHECKSUM 16384 1 ipt_MASQUERADE 16384 3 xt_conntrack 16384 1 ipt_REJECT 16384 2 modinfo provides more information about a specific kernel module, including two interesting sections: the alias and parms. The alias refers to an alternative name that can be used to address the module and, the parms section refer to parameters that can be set while loading the module.\n[root@server1 ~]# modinfo e1000 filename: /lib/modules/4.18.0-193.19.1.el8_2.x86_64/kernel/drivers/net/ethernet/intel/e1000/e1000.ko.xz version: 7.3.21-k8-NAPI license: GPL description: Intel(R) PRO/1000 Network Driver author: Intel Corporation, \u0026lt;linux.nics@intel.com\u0026gt; rhelversion: 8.2 srcversion: 9DFB28D9833DABBB7757EDD alias: pci:v00008086d00002E6Esv*sd*bc*sc*i* ... depends: intree: Y name: e1000 vermagic: 4.18.0-193.19.1.el8_2.x86_64 SMP mod_unload modversions sig_id: PKCS#7 signer: CentOS Linux kernel signing key sig_key: 4C:02:86:8D:9E:A5:E0:4D:A9:C5:DF:8B:D7:28:EA:05:AF:C6:2A:6D sig_hashalgo: sha256 signature: 65:B3:87:34:C5:6F:E5:26:A7:41:90:2C:BB:20:04:54:6E:93:44:2A: 86:73:D7:FF:FD:12:D3:17:74:EB:4B:9B:9C:FB:19:3F:D8:6A:16:10: 0D:72:69:CA:63:B2:2E:63:A9:B4:84:94:0D:4B:C4:94:FC:E6:48:CC: 95:DB:99:65:BC:6F:57:1C:F2:C5:CF:F0:BE:F2:8B:63:11:8F:43:C1: 8C:1C:D3:03:6B:BC:76:0E:18:06:76:F1:C1:CF:72:84:04:92:07:A7: C4:59:4B:7B:72:86:CD:EB:A8:C5:EF:D9:39:FD:B0:38:1A:E3:49:18: 04:88:39:8D:B9:98:D3:5E:EA:0C:CA:B7:44:51:64:F8:7F:CA:01:75: 9A:48:DD:E9:2E:E1:38:60:C6:33:37:1A:81:79:B1:22:63:16:5B:42: DF:E2:08:9B:B4:47:47:9E:9A:69:5D:62:E9:9E:72:A3:7D:D0:E0:B0: 51:24:EA:AD:B1:0B:08:67:63:89:17:19:9A:DF:13:82:FB:C2:DA:32: 97:AA:07:C4:75:A5:6A:A1:E4:AF:D3:64:04:45:24:3F:40:81:21:12: 99:11:54:2C:04:0C:86:98:56:79:C9:34:EC:B9:96:4F:52:BE:A4:CC: 0A:3D:0F:78:5B:0E:1A:E3:7A:57:45:FA:B3:80:EF:B0:2E:75:8F:8B: FE:71:A1:74:63:DC:B2:7E:29:AD:87:4B:6E:AF:66:F7:81:34:1E:0B: 7D:02:71:93:20:01:A7:9B:08:5F:AD:8C:EA:F5:E4:1E:4A:D1:AF:90: CE:23:9A:65:5B:F7:DE:94:3C:DF:6F:5C:15:51:62:D1:64:05:B3:8A: 9A:F4:83:3C:C4:31:E4:EE:A5:6C:0D:56:96:DC:F1:00:53:91:78:BD: D4:20:03:A1:59:07:58:16:B0:8D:7B:19:E6:6A:A3:31:81:7E:31:ED: 77:66:58:B0:F5:68:4E:A0:FA:5C:8B:56:40:4A:BB:77:E3:E3:13:62: 1B:E5:5C:13 parm: TxDescriptors:Number of transmit descriptors (array of int) parm: RxDescriptors:Number of receive descriptors (array of int) parm: Speed:Speed setting (array of int) parm: Duplex:Duplex setting (array of int) parm: AutoNeg:Advertised auto-negotiation setting (array of int) parm: FlowControl:Flow Control setting (array of int) parm: XsumRX:Disable or enable Receive Checksum offload (array of int) parm: TxIntDelay:Transmit Interrupt Delay (array of int) parm: TxAbsIntDelay:Transmit Absolute Interrupt Delay (array of int) parm: RxIntDelay:Receive Interrupt Delay (array of int) parm: RxAbsIntDelay:Receive Absolute Interrupt Delay (array of int) parm: InterruptThrottleRate:Interrupt Throttling Rate (array of int) parm: SmartPowerDownEnable:Enable PHY smart power down (array of int) parm: copybreak:Maximum size of packet that is copied to a new buffer on receive (uint) parm: debug:Debug level (0=none,...,16=all) (int) To manually load and unload modules we use the modprobe and modprobe -r commands. The modprobe command automatically loads any dependencies.\nChecking Driver Availability for Hardware Devices # To check if a particular device is supported and thus has a module loaded you can use the lspci -k command. If there are any devices for which no kernel module was loaded you\u0026rsquo;re likely dealing with an unsupported device.\n[root@server1 ~]# lspci -k 00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller Subsystem: Red Hat, Inc. QEMU Virtual Machine 00:01.0 VGA compatible controller: Red Hat, Inc. Virtio GPU (rev 01) Subsystem: Red Hat, Inc. Device 1100 Kernel driver in use: virtio-pci ... Managing Kernel Module Parameters # You may want to load kernel modules with specific parameters you\u0026rsquo;ve discovered using the modinfo command. To do so, specify the name of the parameter and its value in the modprobe command:\n[root@server1 ~]# modprobe cdrom debug=1 [root@server1 ~]# To make this persistent, you can add an entry to /etc/modprobe.conf or create a file in the /etc/modprobe.d/ directory where the name of the file matches the module name and the content specifies the parameters you want to set:\n[root@server1 modprobe.d]# pwd /etc/modprobe.d [root@server1 modprobe.d]# cat cdrom.conf options cdrom debug=1 Upgrading the Linux Kernel # When upgrading the Linux kernel a new version of the kernel is installed next to the current version and will be used by default. The kernel files for the last four kernels installed will be kept in /boot. The GRUB2 boot loader automatically picks up all kernels found in this directory, allowing you to select an older kernel at boot time in case the newly installed kernel doesn\u0026rsquo;t boot correctly.\nTo install a new version of the kernel, issue the yum upgrade kernel or yum install kernel command.\n","date":"12 May 2021","externalUrl":null,"permalink":"/basic-kernel-management/","section":"Blog","summary":"","title":"Basic Kernel Management","type":"posts"},{"content":"","date":"12 May 2021","externalUrl":null,"permalink":"/tags/kernel-management/","section":"Tags","summary":"","title":"Kernel Management","type":"tags"},{"content":"","date":"6 April 2021","externalUrl":null,"permalink":"/tags/advanced-storage/","section":"Tags","summary":"","title":"Advanced Storage","type":"tags"},{"content":" Virtual Data Optimizer is a storage solution developed to reduce disk space usage on block devices by applying deduplication features. VDO creates volumes on top of any existing block device from where you either create an XFS file system, or use the volume as a Physical Volume in an LVM setup.\nVDO uses three common technologies:\nZero-block elimination to filter out data blocks that contain only zeros. Deduplication of redundant data blocks. Compression when the kvdo module compresses data blocks. Typical usage cases for VDO are host platforms for containers and virtual machines or cloud block storage. Commonly, a logical size of up to 10 times the physical size is used for these types of environments.\nSetting up VDO # To use VDO the underlying block devices must have a minimal size of 4GiB and the vdo and kmod-kvdo packages must be installed.\nWe create the VDO device using the vdo create command, specify a name using the --name= option, and we can specify the logical size using the --vdoLogicalSize= option. e.g. vdo create --name=myvdo1 --vdoLogicalSize=1T /dev/sdb\nOnce the device is created, we can put an XFS file system on top of it: mkfs.xfs -K /dev/mapper/myvdo1 The -K option prevents unused blocks from being discarded immediately, making the command much faster.\nAt this point we issue the udevadm settle command to ensure device nodes have been created succesfully.\nTo persistently mount the VDO file system using the /etc/fstab file we must include the following mount options: x-systemd.requires=vdo.service,discard This makes sure the vdo service is loaded before trying to mount the file system.\nAn alternative method to persistently mount the VDO file system is to use the example systemd mount unit found in /usr/share/doc/vdo/examples/systemd. Copy it to /etc/systemc/system/mountpointname.mount and edit the following lines:\nname = What = Where = The Unit file name must correspond to the name, What and Where values. Make sure to enable and start the moutn at boot: systemctl enable --now mountpointname.mount\nExample # [root@server1 ~]# vdo create --name=vdo1 --device=/dev/sdb --vdoLogicalSize=1T Creating VDO vdo1 The VDO volume can address 2 GB in 1 data slab. It can grow to address at most 16 TB of physical storage in 8192 slabs. If a larger maximum size might be needed, use bigger slabs. Starting VDO vdo1 Starting compression on VDO vdo1 VDO instance 0 volume is ready at /dev/mapper/vdo1 [root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 25G 0 disk ├─sda1 8:1 0 1G 0 part /boot └─sda2 8:2 0 24G 0 part ├─cl-root 253:0 0 22G 0 lvm / └─cl-swap 253:1 0 2.1G 0 lvm [SWAP] sdb 8:16 0 5G 0 disk └─vdo1 253:2 0 1T 0 vdo sdc 8:32 0 5G 0 disk [root@server1 ~]# mkfs.xfs -K /dev/mapper/vdo1 meta-data=/dev/mapper/vdo1 isize=512 agcount=4, agsize=67108864 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=268435456, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=131072, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@server1 ~]# udevadm settle [root@server1 ~]# cp /usr/share/doc/vdo/examples/systemd/VDO.mount.example /etc/systemd/system/vdo1.mount [root@server1 ~]# vim /etc/systemd/system/vdo1.mount .... [root@server1 ~]# cat /etc/systemd/system/vdo1.mount [Unit] Description = Mount filesystem that lives on VDO name = vdo1.mount Requires = vdo.service systemd-remount-fs.service After = multi-user.target Conflicts = umount.target [Mount] What = /dev/mapper/vdo1 Where = /vdo1 Type = xfs Options = discard [Install] WantedBy = multi-user.target [root@server1 ~]# systemctl enable --now vdo1.mount [root@server1 ~]# vdostats --human-readable Device Size Used Available Use% Space saving% /dev/mapper/vdo1 5.0G 3.0G 2.0G 60% 99% [root@server1 ~]# df -h /vdo1/ Filesystem Size Used Avail Use% Mounted on /dev/mapper/vdo1 1.0T 7.2G 1017G 1% /vdo1 [root@server1 ~]# reboot ","date":"6 April 2021","externalUrl":null,"permalink":"/advanced-storage-virtual-data-optimizer/","section":"Blog","summary":"","title":"Advanced Storage: Virtual Data Optimizer","type":"posts"},{"content":"","date":"6 April 2021","externalUrl":null,"permalink":"/tags/vdo/","section":"Tags","summary":"","title":"VDO","type":"tags"},{"content":"","date":"6 April 2021","externalUrl":null,"permalink":"/tags/virtual-data-optimizer/","section":"Tags","summary":"","title":"Virtual Data Optimizer","type":"tags"},{"content":" Stratis, created as an answer to Btrfs and ZFS by Red Hat, is a volume-managing file system that introduces advanced storage features like:\nThin-provisioning: The file system presents itself to users as much bigger than it really is. Useful in virtualized environments. Snapshots: Allows users to backup the current state of the file system and makes it easy to revert to a previous state. Cache tier: A Ceph storage feature that ensures data is stored physically closer to the Ceph client, making data access faster. Programmatic API: Storage can be configured and modified through API access, particularly useful in cloud environments. Monitoring and repair: Stratis has built-in features to monitor and repair the file system, compared to traditional file systems which would rely on tools like fsck. Stratis Architecture # The lowest layer in the Stratis architecture is the pool, which is comparable to an LVM volume group. The pool represents all available storage and consists of one or more storage devices (referred to as blockdev). These block devices can be of any type, including LVM devices but not partitions, but cannot be thin-provisioned as Stratis creates volumes that are thing provisioned themselves. Stratis creates a /stratis/poolname directory for each pool, this directory contains links to devices that represent the file systems in the pool.\nFrom the Stratis pool file systems are created which live in a volume on top of the pool. A pool can contain one or more file systems. Stratis only works with XFS file systems and these are integrated within the Stratis volume: You should not reformat or reconfigure XFS file systems that are managed by Stratis.\nThe file systems are thin-provisioned, they don\u0026rsquo;t have a fixed size and grow automatically as more data is added to the file system.\nCreating and Mounting Stratis Storage # To create Stratis storage, we need to create a pool from a block device and add a file system on top of the pool. Block devices need to be 1GiB at a minimum. Note that a Stratis file systems occupies a minimum of 527MiB even if no data has been added.\nLet\u0026rsquo;s make sure we have the stratis-cli and stratisd packages installed, then start and enable the stratisd daemon:\n[root@server1 ~]# yum install stratis-cli stratisd ... [root@server1 ~]# systemctl enable --now stratisd [root@server1 ~]# systemctl status stratisd ● stratisd.service - A daemon that manages a pool of block devices to create flexible file systems Loaded: loaded (/usr/lib/systemd/system/stratisd.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-08-19 12:25:50 EDT; 4s ago We create a pool from one of the available block devices. Make sure the block device does not contain any file system (use blkid -p /dev/sdx) or partition table, if so we wipe them with the wipefs command, e.g wipefs --all /dev/sdx.\n[root@server1 ~]# stratis pool create mypool1 /dev/sda [root@server1 ~]# stratis pool list Name Total Physical Size Total Physical Used mypool1 10 GiB 52 MiB Next, we create the myfs1 file system on top of the mypool1 pool:\n[root@server1 ~]# stratis fs create mypool1 myfs1 [root@server1 ~]# stratis fs list Pool Name Name Used Created Device UUID mypool1 myfs1 545 MiB Aug 19 2020 12:28 /stratis/mypool1/myfs1 ffdbb3a131f6421c990f69aa8d87c6aa [root@server1 ~]# To peristently mount a Stratis file system, the UUID must be used in the /etc/fstab file and the mount option x-systemd.requires=stratisd.service must be specified to ensure that Systemd waits to activate this device until the stratisd service is loaded:\n[root@server1 ~]# blkid -p /stratis/mypool1/myfs1 /stratis/mypool1/myfs1: UUID=\u0026#34;ffdbb3a1-31f6-421c-990f-69aa8d87c6aa\u0026#34; TYPE=\u0026#34;xfs\u0026#34; USAGE=\u0026#34;filesystem\u0026#34; [root@server1 ~]# mkdir /mnt/myfs1 [root@server1 ~]# vim /etc/fstab ... UUID=ffdbb3a1-31f6-421c-990f-69aa8d87c6aa /mnt/myfs1 xfs defaults,x-systemd.requires=stratisd.service 0 0 ... [root@server1 ~]# [root@server1 ~]# mount -a [root@server1 ~]# reboot Managing Stratis # Traditional Linux tools cannot handle thin-provisioned volumes, we need to use the Stratis specific tools:\nstratis blockdev: Shows information about all block devices. stratis pool: Shows information about Stratis pools. stratis fs: Shows information about file systems. You can use tab-completion on the above commands to reveal specific options.\nExpanding and Renaming a Pool and File System # We can add a block device to a pool to expand the storage capacity of the pool using the stratis pool add-data poolname blockdevice command:\n[root@server1 ~]# stratis pool list Name Total Physical Size Total Physical Used mypool1 10 GiB 597 MiB [root@server1 ~]# stratis pool add-data mypool1 /dev/sdb [root@server1 ~]# stratis pool list Name Total Physical Size Total Physical Used mypool1 15 GiB 601 MiB Destroying a Pool and File System # To destroy a pool and file system, we need to unmount the file system first. Then use the stratis fs destroy poolname fsname command, followed by the stratis pool destroy poolname command:\n[root@server1 ~]# stratis fs list Pool Name Name Used Created Device UUID mypool1 myfs1 545 MiB Aug 19 2020 12:28 /stratis/mypool1/myfs1 ffdbb3a131f6421c990f69aa8d87c6aa [root@server1 ~]# umount /stratis/mypool1/myfs1 [root@server1 ~]# stratis fs destroy mypool1 myfs1 [root@server1 ~]# stratis fs list Pool Name Name Used Created Device UUID [root@server1 ~]# stratis pool list Name Total Physical Size Total Physical Used mypool1 15 GiB 56 MiB [root@server1 ~]# stratis pool destroy mypool1 [root@server1 ~]# stratis pool list Name Total Physical Size Total Physical Used [root@server1 ~]# Creating and Accessing a Stratis Snapshot # In Stratis, a snapshot is a regular Stratis file system created as a copy of another Stratis file system. The snapshot initially contains the same file content as the original file system, but can change as the snapshot is modified. Whatever changes you make to the snapshot will not be reflected in the original file system.\nTo create a Stratis snapshot, use stratis fs snapshot poolname fsname snapshotname. To access the snapshot, mount it as a regular file system from the /stratis/my-pool/ directory: mount /stratis/poolname/snapshotname mount-point\n[root@server1 ~]# stratis pool create mypool1 /dev/sda [root@server1 ~]# stratis pool add-data mypool1 /dev/sdb [root@server1 ~]# stratis pool list Name Total Physical Size Total Physical Used mypool1 15 GiB 56 MiB [root@server1 ~]# stratis fs create mypool1 myfs1 [root@server1 ~]# stratis fs list Pool Name Name Used Created Device UUID mypool1 myfs1 545 MiB Aug 19 2020 13:16 /stratis/mypool1/myfs1 d9e0c47f26e44e0b8990a6aa7546d0f7 [root@server1 ~]# stratis fs snapshot mypool1 myfs1 myfs1snapshot [root@server1 ~]# stratis fs list Pool Name Name Used Created Device UUID mypool1 myfs1 545 MiB Aug 19 2020 13:16 /stratis/mypool1/myfs1 d9e0c47f26e44e0b8990a6aa7546d0f7 mypool1 myfs1snapshot 545 MiB Aug 19 2020 13:17 /stratis/mypool1/myfs1snapshot b2fb662124a4424c9d21429012fcfdc4 [root@server1 ~]# mkdir -p /mnt/myfs1snapshot [root@server1 ~]# mount /stratis/mypool1/myfs1snapshot /mnt/myfs1snapshot/ [root@server1 ~]# umount /mnt/myfs1snapshot [root@server1 ~]# mount /stratis/mypool1/myfs1 /mnt/myfs1 [root@server1 ~]# Reverting a Stratis File System to a Previous Snapshot # It\u0026rsquo;s a good idea to backup the current file system before reverting to a previous snapshot:\n[root@server1 ~]# stratis fs snapshot mypool1 myfs1 myfs1snapshot2 [root@server1 ~]# Next, we unmount and remove the original file system:\n[root@server1 ~]# umount /mnt/myfs1 [root@server1 ~]# stratis fs destroy mypool1 myfs1 [root@server1 ~]# We create a copy of a previous snapshot which we wish to restore, under the name of the original file system:\n[root@server1 ~]# stratis fs list Pool Name Name Used Created Device UUID mypool1 myfs1snapshot2 545 MiB Aug 19 2020 13:23 /stratis/mypool1/myfs1snapshot2 f54d88f686d64acd94c3a7d73dac92f5 mypool1 myfs1snapshot 545 MiB Aug 19 2020 13:17 /stratis/mypool1/myfs1snapshot b2fb662124a4424c9d21429012fcfdc4 [root@server1 ~]# stratis fs snapshot mypool1 myfs1snapshot myfs1 [root@server1 ~]# stratis fs list Pool Name Name Used Created Device UUID mypool1 myfs1snapshot2 545 MiB Aug 19 2020 13:23 /stratis/mypool1/myfs1snapshot2 f54d88f686d64acd94c3a7d73dac92f5 mypool1 myfs1 545 MiB Aug 19 2020 13:31 /stratis/mypool1/myfs1 82f75da64c744079b1c2ae51792812a0 mypool1 myfs1snapshot 545 MiB Aug 19 2020 13:17 /stratis/mypool1/myfs1snapshot b2fb662124a4424c9d21429012fcfdc4 We mount the snapshot, now accessible with the same name as the original file system:\n[root@server1 ~]# mount /stratis/mypool1/myfs1 /mnt/myfs1 [root@server1 ~]# Removing a Stratis Snapshot # We remove a Stratis snapshot by unmounting it first if required, then using the stratis fs destroy poolname snapshotname command.\n[root@server1 ~]# stratis fs destroy mypool1 myfs1snapshot2 [root@server1 ~]# ","date":"28 March 2021","externalUrl":null,"permalink":"/advanced-storage-configuring-stratis/","section":"Blog","summary":"","title":"Advanced Storage: Configuring Stratis","type":"posts"},{"content":"","date":"28 March 2021","externalUrl":null,"permalink":"/tags/stratis/","section":"Tags","summary":"","title":"Stratis","type":"tags"},{"content":" Understanding LVM # The Logical Volume Manager was introduced to workaround some restrictions that come with standard partitions. The most important restriction would be inflexibility, with LVM you can dynamically grow a partition even if the disk itself is running out of space.\nIn the LVM architecture we can distinguish several layers. The lowest layer contains the storage devices, these can be anything from regular disks to partions and logical units (LUNs) on a storage-area network (SAN). Storage devices need to be flagged as physical volumes so that it can be used in an LVM setup. In turn, the physical volume is added to a volume group, which is the abstraction of all available storage space. The volume group can be resized when needed by adding more space (or physical volumes) to the volume group.\nOn top of the volume group we have the logical volumes, they get their disk space from the volume group. This means a logical volume can consist of storage space coming from multiple physical volumes.\nThe actual file systems are created on the logical volumes. The file system must support resizing if the logical volumes are resized.\nWhen running out of disk space on a logical volume, we take available disk space from the volume group. If there is no available disk space on the volume group, we add a physical volume to the volume group.\nThe most important benefit for using LVM would be the added flexibility in managing storage, volumes are not bound to the restrictions of physical hard drives.\nAnother benefit would be the support for snapshots. A snapshot keeps the current state of a logical volume and it can be used to revert to a previous state.\nLVM snapshots are created by copying the logical volume metadata (describing the current state) to a snapshot volume. As long as nothing changes, the original blocks on the volume are addressed. When blocks are modified, the blocks containing the previous state of file are copied to the snapshot volume.\nA third advantage to using LVM would be the option to replace failing hardware easily. If a disk is failing, data can be moved within the volume group, the failing disk can be removed from the volume group and a new disk can be added. This without any downtime for the logical volume itself.\nCreating Logical Volumes # In order to create logical volumes, we need to create the underlying layers in the LVM architecture: First we convert the physical devices into physical volumes, then we create the volume group and assign physical volumes to it, lastly we create the logical volume.\nCreating Physical Volumes # To create a physical volume, we create a partition and mark it with the LVM partition type. For MBR disks that would be type 8e and 8e00 for GUID disks. When using parted you need to use the set n lvm on command, where n is the partition number.\nAfter creating the partition and flagging it as an LVM partition type, we use the pvcreate command to mark it as a physical volume. This writes metadata to the partition so a volume group can use it.\nIn the below example, I\u0026rsquo;ll use an unpartitioned disk (sda) to create a physical volume on.\n[root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 10G 0 disk sr0 11:0 1 1024M 0 rom vda 253:0 0 20G 0 disk ├─vda1 253:1 0 1G 0 part /boot ├─vda2 253:2 0 1G 0 part [SWAP] └─vda3 253:3 0 8G 0 part / [root@server1 ~]# gdisk /dev/sda GPT fdisk (gdisk) version 1.0.3 Partition table scan: MBR: not present BSD: not present APM: not present GPT: not present Creating new GPT entries. Command (? for help): n Partition number (1-128, default 1): First sector (34-20971486, default = 2048) or {+-}size{KMGTP}: Last sector (2048-20971486, default = 20971486) or {+-}size{KMGTP}: +2G Current type is \u0026#39;Linux filesystem\u0026#39; Hex code or GUID (L to show codes, Enter = 8300): 8e00 Changed type of partition to \u0026#39;Linux LVM\u0026#39; Command (? for help): p Disk /dev/sda: 20971520 sectors, 10.0 GiB Model: QEMU HARDDISK Sector size (logical/physical): 512/512 bytes Disk identifier (GUID): 7188B769-C4F4-46C6-8A81-B0E719E621EA Partition table holds up to 128 entries Main partition table begins at sector 2 and ends at sector 33 First usable sector is 34, last usable sector is 20971486 Partitions will be aligned on 2048-sector boundaries Total free space is 16777149 sectors (8.0 GiB) Number Start (sector) End (sector) Size Code Name 1 2048 4196351 2.0 GiB 8E00 Linux LVM Command (? for help): w Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING PARTITIONS!! Do you want to proceed? (Y/N): y OK; writing new GUID partition table (GPT) to /dev/sda. The operation has completed successfully. [root@server1 ~]# pvcreate /dev/sda1 Physical volume \u0026#34;/dev/sda1\u0026#34; successfully created. We can see a summmary of the physical volumes by using the pvs command, or see more details by using the pvdisplay command:\n[root@server1 ~]# pvs PV VG Fmt Attr PSize PFree /dev/sda1 lvm2 --- 2.00g 2.00g [root@server1 ~]# pvdisplay \u0026#34;/dev/sda1\u0026#34; is a new physical volume of \u0026#34;2.00 GiB\u0026#34; --- NEW Physical volume --- PV Name /dev/sda1 VG Name PV Size 2.00 GiB Allocatable NO PE Size 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID 0ata3q-CxXg-WFv6-p3GR-CWxc-wrG3-HxVM6N Creating Volume Groups # Now that we have a physical volume, we should assign it to a volume group. In this case we\u0026rsquo;ll create a new volume group, later on we\u0026rsquo;ll discuss how to add a physical volume to an already existing volume group.\nWe need to issue the vgcreate command followed by the name of the volume group and the name of the physical device we want to add to it:\n[root@server1 ~]# vgcreate vgdata /dev/sda1 Volume group \u0026#34;vgdata\u0026#34; successfully created Check the volume group with the vgs and vgdisplay commands:\n[root@server1 ~]# vgs VG #PV #LV #SN Attr VSize VFree vgdata 1 0 0 wz--n- \u0026lt;2.00g \u0026lt;2.00g [root@server1 ~]# vgdisplay --- Volume group --- VG Name vgdata System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size \u0026lt;2.00 GiB PE Size 4.00 MiB Total PE 511 Alloc PE / Size 0 / 0 Free PE / Size 511 / \u0026lt;2.00 GiB VG UUID h50RrD-QmTD-yY3f-gDiy-TolW-GR1B-gFoaCe We could have created the physical and volume group in one step as long as the partition is marked as an LVM partition. When issuing the command vgcreate vgdata /dev/sda1 without having created the physical volume, the vgcreate utility will automatically flag the partition as a physical volume. This is useful for adding a complete disk device instead of a partition. A complete disk device does not need to be flagged for LVM use in a partion utility, e.g. vgcreate vgbackup /dev/sdb.\nWhen working with LVM there is the physical extent size to consider. This is the size of the basic building blocks used in the LVM configuration. The default extent size is 4.00MiB:\n[root@server1 ~]# vgdisplay | grep \u0026#39;PE\u0026#39; PE Size 4.00 MiB Total PE 511 Alloc PE / Size 0 / 0 Free PE / Size 511 / \u0026lt;2.00 GiB The PE Size is always specified as multiples of 2MiB with a maximum of 128MiB. The vgcreate -s option allows you to specify the PE Size you want to use. If you need to create huge logical volumes it is more efficient to use a big PE Size.\nAbove we can see that the PE Size is 4.00MiB and we have a total PE of 511 blocks. 511 multiplied by 4 would be 2044MiB.\nCreating the Logical Volumes and File Systems # When creating the logical volume, we have to specify a logical volume name and size. We specify the name use the lvcreate -n option, an absolute size with the -L option or a relative size using the -l option:\nlvcreate -n mylvol1 -L 2G vgdata - Creates a logical volume with the name mylvol1 and an absolute size of 2GiB taken from the vgdata volume group. lvcreate -n mylvol1 -l 100%FREE vgdata - Creates a logical volume spanning all available space in the vgdata volume group. lvcreate -n mylvol1 -l 50%FREE vgdata - Creates a logical volume spanning 50% of the available space in the vgdata volume group. [root@server1 ~]# lvcreate -n lvol1 -l 50%FREE vgdata Logical volume \u0026#34;lvol1\u0026#34; created. [root@server1 ~]# lvcreate -n lvol2 -l 100%FREE vgdata Logical volume \u0026#34;lvol2\u0026#34; created. [root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 10G 0 disk └─sda1 8:1 0 2G 0 part ├─vgdata-lvol1 252:0 0 1020M 0 lvm └─vgdata-lvol2 252:1 0 1G 0 lvm [root@server1 ~]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert lvol1 vgdata -wi-a----- 1020.00m lvol2 vgdata -wi-a----- 1.00g [root@server1 ~]# vgs VG #PV #LV #SN Attr VSize VFree vgdata 1 2 0 wz--n- \u0026lt;2.00g 0 Notice how I created one logical volume consisting of half of the available space in the volume group first, then created a second one with the -l 100%FREE option to take all remaining available space.\nAt this point we\u0026rsquo;re ready to create a file system on both logical volumes:\n[root@server1 ~]# mkfs.xfs /dev/vgdata/lvol1 meta-data=/dev/vgdata/lvol1 isize=512 agcount=4, agsize=65280 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=261120, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=1566, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@server1 ~]# mkfs.xfs /dev/vgdata/lvol2 meta-data=/dev/vgdata/lvol2 isize=512 agcount=4, agsize=65536 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=262144, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 Understanding Device Naming # Logical volumes can be addressed in different ways:\n/dev/[volume group]/[logical volume] e.g. /dev/vgdata/lvol1 /dev/mapper/[volumge group]-[logical volume] e.g. /dev/mapper/vgdata-lvol1 The first method is basically a symbolic link to the device mapper (abbreviate as dm), which in turn is a generic interface the Linux kernel uses to address storage devices. Device mapper devices are generated on detection and use meaningless names like /dev/dm-0 and /dev/dm-1:\n[root@server1 ~]# ls -l /dev/vgdata/ total 0 lrwxrwxrwx. 1 root root 7 Aug 14 00:10 lvol1 -\u0026gt; ../dm-0 lrwxrwxrwx. 1 root root 7 Aug 14 00:10 lvol2 -\u0026gt; ../dm-1 To provide easier access there are symbolic links to those same device mapper devices in /dev/mapper:\n[root@server1 ~]# ls -l /dev/mapper/ total 0 crw-------. 1 root root 10, 236 Aug 14 00:10 control lrwxrwxrwx. 1 root root 7 Aug 14 00:10 vgdata-lvol1 -\u0026gt; ../dm-0 lrwxrwxrwx. 1 root root 7 Aug 14 00:10 vgdata-lvol2 -\u0026gt; ../dm-1 When working with LVM logical volumes, you can use either of these device names.\nEssential Commands for LVM Management # Command Description pvcreate Creates physical volumes pvs Summary of available physical volumes pvdisplay List physical volumes and their properties pvremove Removes the physical volume signature from a block device vgcreate Creates a volume group vgs Summary of available volume groups vgdisplay List volume groups and their properties vgremove Removes a volume group lvcreate Creates logical volumes lvs Shows a summary of all available logical volumes lvdisplay List available logical volumes and their properties lvremove Removes a logical volumes Resizing LVM Logical Volumes # The ability to resize logical volumes is one of the major benefits of using LVM.\nWhen using the XFS file system, a volume can be increased in size, but not decreased. Ext4 supports decreasing the file system size, but it must be done when the file system is offline. In other words, you must unmount it before you can resize it. To increase the size of a logical volume, we need to have disk space available in the volume group, so we need to address that first.\nResizing Volume Groups # The vgextend command is used to add storage to a volume group, while the vgreduce command is used to take physical volumes out of a volume group:\nMake sure a physical volume or device is available to be added to a volume group Use the vgextend command to extend the volume group with the space from the new physical volume or device. Use the vgs or vgdisplay commands to verify that a physical volume has been added to the volume group. The number of physical volumes for a specific volume group are indicated in the #PV column. Resizing Logical Volumes and File Systems # Logical volumes can be extended using the lvextend or lvresize commands and it can automatically take care of extending the file system on top using the -r option.\nSimilarly to creating logical volumes, you can extend the logical volume size with the -L or -l options (absolute or relative sizes). If you specify an absolute size, the -L option must be followed by a + sign and the amount of disk space you want to add, e.g. lvextend -L +1G -r /dev/vgdata/lvol1.\nRelative sizes can be specified like so:\nlvextend -r -l 75%VG /dev/vgdata/lvol1 - This resizes the logical volume so that it takes 75% of the total disk space in the volume group.\nlvresize -r -l +75%VG /dev/vgdata/lvol1 - This adds 75% of the total size of the volume group to the logical volume.\nlvextend -r -l +75%FREE /dev/vgdata/lvol1 - Add 75% of all free disk space to the logical volume.\nlvresize -r -l 75%FREE /dev/vgdata/lvol1 - This resizes the logical volume so that it takes 75% of the free disk space in the volume group.\nLet\u0026rsquo;s check the current block devices:\n[root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 10G 0 disk └─sda1 8:1 0 2G 0 part ├─vgdata-lvol1 252:0 0 1020M 0 lvm └─vgdata-lvol2 252:1 0 1G 0 lvm sr0 11:0 1 1024M 0 rom vda 253:0 0 20G 0 disk ├─vda1 253:1 0 1G 0 part /boot ├─vda2 253:2 0 1G 0 part [SWAP] └─vda3 253:3 0 8G 0 part / We create a new partition and mark it as a physical volume:\n[root@server1 ~]# gdisk /dev/sda GPT fdisk (gdisk) version 1.0.3 Partition table scan: MBR: protective BSD: not present APM: not present GPT: present Found valid GPT with protective MBR; using GPT. Command (? for help): p Disk /dev/sda: 20971520 sectors, 10.0 GiB Model: QEMU HARDDISK Sector size (logical/physical): 512/512 bytes Disk identifier (GUID): 7188B769-C4F4-46C6-8A81-B0E719E621EA Partition table holds up to 128 entries Main partition table begins at sector 2 and ends at sector 33 First usable sector is 34, last usable sector is 20971486 Partitions will be aligned on 2048-sector boundaries Total free space is 16777149 sectors (8.0 GiB) Number Start (sector) End (sector) Size Code Name 1 2048 4196351 2.0 GiB 8E00 Linux LVM Command (? for help): n Partition number (2-128, default 2): First sector (34-20971486, default = 4196352) or {+-}size{KMGTP}: Last sector (4196352-20971486, default = 20971486) or {+-}size{KMGTP}: +3G Current type is \u0026#39;Linux filesystem\u0026#39; Hex code or GUID (L to show codes, Enter = 8300): 8e00 Changed type of partition to \u0026#39;Linux LVM\u0026#39; Command (? for help): p Disk /dev/sda: 20971520 sectors, 10.0 GiB Model: QEMU HARDDISK Sector size (logical/physical): 512/512 bytes Disk identifier (GUID): 7188B769-C4F4-46C6-8A81-B0E719E621EA Partition table holds up to 128 entries Main partition table begins at sector 2 and ends at sector 33 First usable sector is 34, last usable sector is 20971486 Partitions will be aligned on 2048-sector boundaries Total free space is 10485693 sectors (5.0 GiB) Number Start (sector) End (sector) Size Code Name 1 2048 4196351 2.0 GiB 8E00 Linux LVM 2 4196352 10487807 3.0 GiB 8E00 Linux LVM Command (? for help): w Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING PARTITIONS!! Do you want to proceed? (Y/N): y OK; writing new GUID partition table (GPT) to /dev/sda. Warning: The kernel is still using the old partition table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) The operation has completed successfully. [root@server1 ~]# partprobe Check the output of lsblk for the new partition:\n[root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 10G 0 disk ├─sda1 8:1 0 2G 0 part │ ├─vgdata-lvol1 252:0 0 1020M 0 lvm │ └─vgdata-lvol2 252:1 0 1G 0 lvm └─sda2 8:2 0 3G 0 part sr0 11:0 1 1024M 0 rom vda 253:0 0 20G 0 disk ├─vda1 253:1 0 1G 0 part /boot ├─vda2 253:2 0 1G 0 part [SWAP] └─vda3 253:3 0 8G 0 part / Check the output of vgs, extend the volume group and check the changes:\n[root@server1 ~]# vgs VG #PV #LV #SN Attr VSize VFree vgdata 1 2 0 wz--n- \u0026lt;2.00g 0 [root@server1 ~]# vgextend vgdata /dev/sda2 Physical volume \u0026#34;/dev/sda2\u0026#34; successfully created. Volume group \u0026#34;vgdata\u0026#34; successfully extended [root@server1 ~]# vgs VG #PV #LV #SN Attr VSize VFree vgdata 2 2 0 wz--n- 4.99g \u0026lt;3.00g Extend the logical volume with all available disk space (3GiB) from the volume group:\n[root@server1 ~]# lvresize -r -l +100%FREE /dev/vgdata/lvol1 Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan (but don\u0026#39;t clear) agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 No modify flag set, skipping phase 5 Phase 6 - check inode connectivity... - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify link counts... No modify flag set, skipping filesystem flush and exiting. Size of logical volume vgdata/lvol1 changed from \u0026lt;3.00 GiB (767 extents) to 3.99 GiB (1022 extents). Logical volume vgdata/lvol1 successfully resized. meta-data=/dev/mapper/vgdata-lvol1 isize=512 agcount=13, agsize=65280 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=785408, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=1566, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 785408 to 1046528 Notice that I was using the XFS file system and that the lvresize -r option resized the file system as well.\nVerify the changes in lbsblk:\n[root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 10G 0 disk ├─sda1 8:1 0 2G 0 part │ ├─vgdata-lvol1 252:0 0 4G 0 lvm │ └─vgdata-lvol2 252:1 0 1G 0 lvm └─sda2 8:2 0 3G 0 part └─vgdata-lvol1 252:0 0 4G 0 lvm sr0 11:0 1 1024M 0 rom vda 253:0 0 20G 0 disk ├─vda1 253:1 0 1G 0 part /boot ├─vda2 253:2 0 1G 0 part [SWAP] └─vda3 253:3 0 8G 0 part / ","date":"6 March 2021","externalUrl":null,"permalink":"/advanced-storage-logical-volume-manager/","section":"Blog","summary":"","title":"Advanced Storage: Logical Volume Manager","type":"posts"},{"content":"","date":"6 March 2021","externalUrl":null,"permalink":"/tags/logical-volume-manager/","section":"Tags","summary":"","title":"Logical Volume Manager","type":"tags"},{"content":"","date":"6 March 2021","externalUrl":null,"permalink":"/tags/lvm/","section":"Tags","summary":"","title":"LVM","type":"tags"},{"content":"","date":"4 February 2021","externalUrl":null,"permalink":"/tags/file-systems/","section":"Tags","summary":"","title":"File Systems","type":"tags"},{"content":" Partitions by themselves aren\u0026rsquo;t of any use if they don\u0026rsquo;t contain a file system. On RHEL 8 different file systems can be used, the default being xfs. To format a partition with one of the supported file systems we can use the mkfs command followed by the -t option to specify a specific file system. Alternatively, you can use a file system-specific tool like mkfs.xfs to format an xfs file system or mkfs.ext4 to format an Ext4 file system.\nIf you use mkfs without specifying what file system to format, the Ext2 file system is used.\nAmong the supported file systems, we can distuinguish Journaling, Non-Journaling and FAT File Systems:\nNon-Journaling:\nExt2 - Extended File System 2, Legacy Linux file system. There is no use for this on RHEL 8. Journaling: Uses a journal to keep track of changes that have not been written to the file system. This provides some protection for file corruption during system crashes and unexpected shutdowns\nExt3 - Previous version of Ext4. There is no use for this on RHEL 8. Ext4 - The default file system in previous versions of RHEL. Still supported on RHEL 8. XFS - The default RHEL 8 file system. BTRFS: Uses Copy on Write (CoW), a resource management technique. Uses Subvolumes: Similar to a partition, but can be accessed like a directory. Snapshots: Subvolumes that reference the original data\u0026rsquo;s location, metadata and directory structure. File Allocation Table file systems\nLinux can use VFAT (Virtual File Allocation TAble) which allows longer file names. EFI Boot partitions need to use a FAT partition (VFAT on Linux) ex-FAT - Extended FAT file system: Allows files larger than 2Gb. [root@server1 ~]# mkfs -t xfs /dev/vda3 meta-data=/dev/vda3 isize=512 agcount=4, agsize=131072 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=524288, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 Remember you can\u0026rsquo;t format a file system onto an Extended partition, it contains the partition table for the Logical partitions.\n[root@server1 ~]# mkfs.xfs /dev/vda4 mkfs.xfs: /dev/vda4 appears to contain a partition table (dos). mkfs.xfs: Use the -f option to force overwrite. [root@server1 ~]# mkfs.xfs /dev/vda5 meta-data=/dev/vda5 isize=512 agcount=4, agsize=262144 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=1048576, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 Changing File System Properties # File system properties can be managed using different tools specific for the file system you\u0026rsquo;re using.\nManaging Ext4 File System Properties # The generic tool for managing Ext4 file system properties is tune2fs. This tool was developped for Ext2 but is fully compatible with Ext3 and Ext4.\ntune2fs -l shows the different file system properties:\n[root@server1 ~]# tune2fs -l /dev/vdb1 tune2fs 1.44.6 (5-Mar-2019) Filesystem volume name: \u0026lt;none\u0026gt; Last mounted on: \u0026lt;not available\u0026gt; Filesystem UUID: 492d646a-950d-49da-a3f3-62f0872a9f4b Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum Filesystem flags: signed_directory_hash Default mount options: user_xattr acl Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 655360 Block count: 2621179 Reserved block count: 131058 Free blocks: 2554426 Free inodes: 655349 First block: 0 Block size: 4096 Fragment size: 4096 Group descriptor size: 64 Reserved GDT blocks: 1024 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Flex block group size: 16 Filesystem created: Fri Jul 31 20:51:26 2020 Last mount time: n/a Last write time: Fri Jul 31 21:01:22 2020 Mount count: 0 Maximum mount count: -1 Last checked: Fri Jul 31 20:51:26 2020 Check interval: 0 (\u0026lt;none\u0026gt;) Lifetime writes: 68 MB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 32 Desired extra isize: 32 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: 5613adcb-e707-4a6d-8559-53c1f238609d Journal backup: inode blocks Checksum type: crc32c Checksum: 0x4828a56a Interesting properties are the file system label (showing as Filesystem volume name), File System features and Default mount option:\nUse tune2fs -o to set default file system mount option: tune2fs -o acl,user_xatrr /dev/vdb1 to switch on access control lists and user extended attributes. tune2fs -o ^acl,user_xattr /dev/vdb1 to switch off the same options. Use tune2fs -O to set or unset file system features: tune2fs -O ^dir_index /dev/vdb1 tune2fs -O dir_index /dev/vdb1 Use tune2fs -L or e2label2 to set a file system label: tune2fs -L MyData /dev/vdb1 Use tune2fs -i to set file system checks intervals: tune2fs -i 3w /dev/vdb1 sets the interval to 1814400 seconds, or 3 weeks. File system labels or volume names will come in handy when mounting file systems. We can use the label instead of the device name for consistent mounting, even if the underlying device name changes.\nManaging XFS File System Properties # The XFS file system is completely different, you can not set file system attributes within the file system metadata. You can however change some XFS properties, like the volume label, using the xfs_admin command:\n[root@server1 ~]# xfs_admin -L XFS_Disk /dev/vda3 writing all SBs new label = \u0026#34;XFS_Disk\u0026#34; [root@h ~]# blkid | grep vda3 /dev/vda3: LABEL=\u0026#34;XFS_Disk\u0026#34; UUID=\u0026#34;1e235236-8dfa-42d3-9948-826df137780c\u0026#34; TYPE=\u0026#34;xfs\u0026#34; PARTUUID=\u0026#34;de6faae3-03\u0026#34; [root@server1 ~]# xfs_admin -l /dev/vda3 label = \u0026#34;XFS_Disk\u0026#34; Adding Swap Files and Partitions # Using swap on Linux is a convenient way to improve kernel memory usage. If a shortage of physical RAM occurs, non-recently used memory pages can be moved to swap space to make more RAM available for other programs. However, intensive usage of swap space could indicate a potential problem, swap space should be closely monitored.\nSwap space is either created by formatting a partition with the swap partition type, or creating a swap file and formatting the file as swap space. From a performance point of view it doesn\u0026rsquo;t make much difference if a swap partition or a swap file is being used. A swap file could be helpful if you need to increase swap space, but you don\u0026rsquo;t have free disk space to create a partition.\nFor a swap partition, use fdisk or gdisk depending on the partition table:\n[root@server1 ~]# fdisk /dev/vda Welcome to fdisk (util-linux 2.32.1). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): p Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xde6faae3 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 37750783 37748736 18G 83 Linux /dev/vda2 37750784 41945087 4194304 2G 82 Linux swap / Solaris Command (m for help): n Partition type p primary (2 primary, 0 extended, 2 free) e extended (container for logical partitions) Select (default p): p Partition number (3,4, default 3): First sector (41945088-62914559, default 41945088): Last sector, +sectors or +size{K,M,G,T,P} (41945088-62914559, default 62914559): +2G Created a new partition 3 of type \u0026#39;Linux\u0026#39; and of size 2 GiB. Command (m for help): t Partition number (1-3, default 3): Hex code (type L to list all codes): 82 Changed type of partition \u0026#39;Linux\u0026#39; to \u0026#39;Linux swap / Solaris\u0026#39;. Command (m for help): w The partition table has been altered. Syncing disks. [root@server1 ~]# mkswap /dev/vda3 Setting up swapspace version 1, size = 2 GiB (2147479552 bytes) no label, UUID=61fc53d7-f385-4c79-94bd-69b33010c09a [root@server1 ~]# free -m total used free shared buff/cache available Mem: 1829 1015 191 21 622 642 Swap: 2047 0 2047 [root@server1 ~]# swapon /dev/vda3 [root@server1 ~]# free -m total used free shared buff/cache available Mem: 1829 1017 189 21 622 640 Swap: 4095 0 4095 [root@server1 ~]# To add a swap file, create the file first. The below dd command creates a file containing 512 blocks of 1MiB containing all zeroes.\n[root@server1 ~]# dd if=/dev/zero of=/swapfile bs=1M count=512 512+0 records in 512+0 records out 536870912 bytes (537 MB, 512 MiB) copied, 6.28219 s, 85.5 MB/s [root@server1 ~]# chmod 0600 /swapfile [root@server1 ~]# mkswap /swapfile Setting up swapspace version 1, size = 512 MiB (536866816 bytes) no label, UUID=0f2316f3-d713-4752-be6b-74584092c193 [root@server1 ~]# swapon /swapfile [root@server1 ~]# free -m total used free shared buff/cache available Mem: 1829 1002 170 12 655 664 Swap: 4607 35 4572 Mounting File Systems # To use a partition we have to mount it to make its content available through a specific directory. In order to do so, we need specific information:\nWhat - Mandatory information that specifies what device we want to mount. Where - Mandatory information that specifies the directory on wich we want to mount our device. File System - Optional. Typically the mount command will detect the proper file system. Options - Optional but depends on the needs you have for the file system. To mount a file system, the mount command is used, to unmount a file system the umount command is used:\n[root@server1 ~]# mkdir /ext4dir [root@server1 ~]# mount /dev/vdb1 /ext4dir [root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 7G 0 rom vda 253:0 0 30G 0 disk ├─vda1 253:1 0 18G 0 part / ├─vda2 253:2 0 2G 0 part [SWAP] └─vda3 253:3 0 2G 0 part [SWAP] vdb 253:16 0 10G 0 disk └─vdb1 253:17 0 10G 0 part /ext4dir [root@server1 ~]# umount /ext4dir [root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 7G 0 rom vda 253:0 0 30G 0 disk ├─vda1 253:1 0 18G 0 part / ├─vda2 253:2 0 2G 0 part [SWAP] └─vda3 253:3 0 2G 0 part [SWAP] vdb 253:16 0 10G 0 disk └─vdb1 253:17 0 10G 0 part [root@server1 ~]# Note that for unmounting, both umount /dev/vdb1 or umount /ext4dir would work.\nDevice Names, UUIDs, or Disk Labels # Typically we use device names like /dev/vdb1 to mount devices. If you\u0026rsquo;re in a environment where a dynamic storage topology is being used this is not a good approach, since device names may change after e.g. a server reboot.\nOn a default RHEL 8 installation, Universally Unique Identifiers are used for every file system. This may not be handy when manually mounting, but does make sense when automating file system mounts. Before the use of UUIDs was common, file systems were often mounted via their Volume Name or Disk Label. Both the UUID and the Disk Label can be seen using the blkid command:\n[root@server1 ~]# blkid /dev/vda1: UUID=\u0026#34;658aed1c-058c-42a6-bce1-ffcf41426b53\u0026#34; TYPE=\u0026#34;xfs\u0026#34; PARTUUID=\u0026#34;de6faae3-01\u0026#34; /dev/vda2: UUID=\u0026#34;c8ad4b20-2469-4caf-aa30-1a99d16870d9\u0026#34; TYPE=\u0026#34;swap\u0026#34; PARTUUID=\u0026#34;de6faae3-02\u0026#34; /dev/vda3: UUID=\u0026#34;61fc53d7-f385-4c79-94bd-69b33010c09a\u0026#34; TYPE=\u0026#34;swap\u0026#34; PARTUUID=\u0026#34;de6faae3-03\u0026#34; /dev/vdb1: LABEL=\u0026#34;ext4disk\u0026#34; UUID=\u0026#34;492d646a-950d-49da-a3f3-62f0872a9f4b\u0026#34; TYPE=\u0026#34;ext4\u0026#34; PARTLABEL=\u0026#34;Linux filesystem\u0026#34; PARTUUID=\u0026#34;e8063de5-2db0-455b-a50b-a3b2f69e857c\u0026#34; [root@server1 ~]# mount UUID=\u0026#34;492d646a-950d-49da-a3f3-62f0872a9f4b\u0026#34; /ext4dir [root@server1 ~]# umount /ext4dir [root@server1 ~]# mount LABEL=\u0026#34;ext4disk\u0026#34; /ext4dir [root@server1 ~]# Automating File System Mounts Through /etc/fstab # Usually you would want to mount file systems automatically, the classical way of doing this is through the /etc/fstab file. This file specifies everything needed to mount file systems:\n[root@server1 ~]# cat /etc/fstab # # /etc/fstab # Created by anaconda on Mon Jun 22 03:30:44 2020 # # Accessible filesystems, by reference, are maintained under \u0026#39;/dev/disk/\u0026#39;. # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. # # After editing this file, run \u0026#39;systemctl daemon-reload\u0026#39; to update systemd # units generated from this file. # UUID=658aed1c-058c-42a6-bce1-ffcf41426b53 / xfs defaults 0 0 UUID=c8ad4b20-2469-4caf-aa30-1a99d16870d9 swap swap defaults 0 0 Each line in the file constains six fields:\nField Description Device A device name, UUID or label. Mount Point A directory or kernel interface where the device needs to be mounted. File System The file system type. Mount Options The mount options applied to the file system. Dump Support Set to 1 to enable support using the dump utility. Necessary for some backup solutions. Automatic Check File system integrity check. 0 to disable automated checks, 1 if this is the root file system to be checked automatically, 2 for all other file systems that need automatic checking while booting. Network file systems should have this option set to 0. The xfs file system does not support file system checks, so in this case the Automatic Check field should be set to 0.\nNot all file systems use a directory as a mount point, e.g. system devices like swap use a kernel interface. You can easily recognize kernel interfaces, their name doesn\u0026rsquo;t start with a / like a directory (and it doesn\u0026rsquo;t exist in the file system).\nThe Mount Options field defines specific moutn options, if no options are required then this field will read \u0026ldquo;defaults\u0026rdquo;. The following common mount options can be used:\nOption Description auto/noauto The file system will (not) be mounted automatically. acl Adds support for Access Control Lists. user_xattr Adds support for user-extended attributes. ro Mounts the file system in read only mode. atime/noatime Enables or disables access time modifications. exec/noexec Allows or denies execution of program files from the file system. _netdev To mount a network file system. This tells fstab to wait until the network is available before mounting. [root@localhost ~]# cat /etc/fstab # # /etc/fstab # Created by anaconda on Mon Jun 22 03:30:44 2020 # # Accessible filesystems, by reference, are maintained under \u0026#39;/dev/disk/\u0026#39;. # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. # # After editing this file, run \u0026#39;systemctl daemon-reload\u0026#39; to update systemd # units generated from this file. # UUID=658aed1c-058c-42a6-bce1-ffcf41426b53 / xfs defaults 0 0 UUID=c8ad4b20-2469-4caf-aa30-1a99d16870d9 swap swap defaults 0 0 UUID=492d646a-950d-49da-a3f3-62f0872a9f4b /ext4dir ext4 defaults 0 0 /swapfile swap swap defaults 0 0 [root@localhost ~]# mount -a [root@localhost ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 7G 0 rom /run/media/student/CentOS-8-1-1911-x86_64-dvd vda 253:0 0 30G 0 disk ├─vda1 253:1 0 18G 0 part / ├─vda2 253:2 0 2G 0 part [SWAP] └─vda3 253:3 0 2G 0 part [SWAP] vdb 253:16 0 10G 0 disk └─vdb1 253:17 0 10G 0 part /ext4dir ","date":"4 February 2021","externalUrl":null,"permalink":"/managing-storage-creating-mounting-file-systems/","section":"Blog","summary":"","title":"Managing Storage: Creating \u0026 Mounting File Systems","type":"posts"},{"content":"","date":"4 February 2021","externalUrl":null,"permalink":"/tags/mount/","section":"Tags","summary":"","title":"Mount","type":"tags"},{"content":"","date":"29 January 2021","externalUrl":null,"permalink":"/tags/fdisk/","section":"Tags","summary":"","title":"Fdisk","type":"tags"},{"content":"","date":"29 January 2021","externalUrl":null,"permalink":"/tags/gdisk/","section":"Tags","summary":"","title":"Gdisk","type":"tags"},{"content":"","date":"29 January 2021","externalUrl":null,"permalink":"/tags/gpt/","section":"Tags","summary":"","title":"GPT","type":"tags"},{"content":" To match the different partition types we use different partitioning utilities. The fdisk utlity is used to create MBR partitions while the gdisk utility is used to create GPT paritions.\nApart from fdisk and gdisk, there is the parted command which can create both MBR and GPT partitions but has less advanced features.\nEach command takes a disk device name as an argument. The device names are usually /dev/sda, /dev/sdb, \u0026hellip; in the order the device is recognized by the kernel. You can have disks up to /dev/sdz and beyond: /dev/sdaa, /dev/sdab, \u0026hellip;\nPartitions are numbered, the /dev/sda device contains partitions like /dev/sda1 , /dev/sda2, \u0026hellip;\nThe name of the device also depends on the type of driver that\u0026rsquo;s used:\nDevice Name Description /dev/sda Devices that use the SCSI driver, used for both SCSI and SATA devices. This is common on physical machines but also on VMware virtual machines. /dev/nvme0n1 The first hard disk on an NVME Express interface. Note that the first drive is referred to as n1 instead of a. /dev/hda Legacy IDE disk devices. /dev/vda Common on KVM virtual machines using the virtio disk driver. /dev/xvda A disk in a Xen virtual machine that uses the Xen virtual disk driver. Creating MBR Partitions with fdisk # Next, I will show you how to use fdisk to create a partition on nonpartitioned disk space. If you don\u0026rsquo;t have nonpartitioned disk space the same principle can be applied to a separate virtual disk. Make sure to create a snapshot of your virtual machine so you can easily revert back.\nThe lsblk command shows me I have two disks: vda and vdb, where vda contains two partitions:\n[root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 7G 0 rom vda 253:0 0 30G 0 disk ├─vda1 253:1 0 18G 0 part / └─vda2 253:2 0 2G 0 part [SWAP] vdb 253:16 0 10G 0 disk df -h shows me the available disk space on each disk, for /dev/vda that would be 7.2Gb. There\u0026rsquo;s no file system yet on /dev/vdb so it\u0026rsquo;s not showing anything for this disk.\n[root@server1 ~]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 899M 0 899M 0% /dev tmpfs 915M 0 915M 0% /dev/shm tmpfs 915M 1.6M 914M 1% /run tmpfs 915M 0 915M 0% /sys/fs/cgroup /dev/vda1 18G 11G 7.2G 61% / tmpfs 183M 12K 183M 1% /run/user/42 tmpfs 183M 28K 183M 1% /run/user/1000 Let\u0026rsquo;s run the fdisk command against the /dev/vda disk and print out the current disk allocation:\n[root@server1 ~]# fdisk /dev/vda Welcome to fdisk (util-linux 2.32.1). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): p Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xde6faae3 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 37750783 37748736 18G 83 Linux /dev/vda2 37750784 41945087 4194304 2G 82 Linux swap / Solaris Command (m for help): We can see that this disk has 62914560 sectors and that the last partition does not end on the last sector This confirms again we have free disk space available to create a new partition.\nType n to create a new partition, followed by p to create a primary partition. You can choose the new parition number or hit enter to accept the default.\nCommand (m for help): n Partition type p primary (2 primary, 0 extended, 2 free) e extended (container for logical partitions) Select (default p): p Partition number (3,4, default 3): 3 fdisk will suggest the first available sector, you can accept that by hitting enter. For the last sector (which will ultimately determine the size), you can choose to accept the default to create a partition on all remaining disk space, or you can specify the last sector (using +numberofsectors) or specify a size using +number(K,M,G), i.e. +1G will make the partition 1GiB in size.\nFirst sector (41945088-62914559, default 41945088): Last sector, +sectors or +size{K,M,G,T,P} (41945088-62914559, default 62914559): +1G Created a new partition 3 of type \u0026#39;Linux\u0026#39; and of size 1 GiB. By default a Linux parition type is used. You can change the type using the t command. In this case we\u0026rsquo;ll continue to use the Linux type.\nUse the p command to print the disk allocation again:\nCommand (m for help): p Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xde6faae3 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 37750783 37748736 18G 83 Linux /dev/vda2 37750784 41945087 4194304 2G 82 Linux swap / Solaris /dev/vda3 41945088 44042239 2097152 1G 83 Linux Once we\u0026rsquo;re happy with the modifications, we use w to write them to disk and exit fdisk.\n(Type q at anytime if you need to quit fdisk without writing your changes.)\nCommand (m for help): w The partition table has been altered. Syncing disks. It\u0026rsquo;s possible you\u0026rsquo;ll receive the following warning message:\nWARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8). This means the partition table was written succesfully, but the in-memory kernel partition could not be updated. You can check this by comparing the output of fdisk -l /dev/vda with cat /proc/partitions.\nRun partprobe /dev/vda to write the changes to the in-memory kernel parition table. Typically this happens when you add partitions to a disk that already has mounted partitions.\nUsing Extended and Logical Partitions on MBR # If three partitions have been created already, there is room for 1 more primary partition. If you need to go beyond 4 partitions on an MBR disk you will have to create an extended partition and create logical partitions within it.\nIf something goes wrong with your extended partition you will have a problem with all logical partitions as well. With this in mind you might be better off using LVM, which I\u0026rsquo;ll discuss later on in a different post.\nAn extended partition is used only for the purpose of creating logical partitions. Consider it as a container for the logical partitions, you can\u0026rsquo;t create a filesystem directly on the extended partition.\n[root@server1 ~]# fdisk /dev/vda Welcome to fdisk (util-linux 2.32.1). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): p Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xde6faae3 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 37750783 37748736 18G 83 Linux /dev/vda2 37750784 41945087 4194304 2G 82 Linux swap / Solaris /dev/vda3 41945088 44042239 2097152 1G 83 Linux Command (m for help): n Partition type p primary (3 primary, 0 extended, 1 free) e extended (container for logical partitions) Select (default e): e Selected partition 4 First sector (44042240-62914559, default 44042240): Last sector, +sectors or +size{K,M,G,T,P} (44042240-62914559, default 62914559): Created a new partition 4 of type \u0026#39;Extended\u0026#39; and of size 9 GiB. Command (m for help): p Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xde6faae3 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 37750783 37748736 18G 83 Linux /dev/vda2 37750784 41945087 4194304 2G 82 Linux swap / Solaris /dev/vda3 41945088 44042239 2097152 1G 83 Linux /dev/vda4 44042240 62914559 18872320 9G 5 Extended Command (m for help): n All primary partitions are in use. Adding logical partition 5 First sector (44044288-62914559, default 44044288): Last sector, +sectors or +size{K,M,G,T,P} (44044288-62914559, default 62914559): +5G Created a new partition 5 of type \u0026#39;Linux\u0026#39; and of size 5 GiB. Command (m for help): n All primary partitions are in use. Adding logical partition 6 First sector (54532096-62914559, default 54532096): Last sector, +sectors or +size{K,M,G,T,P} (54532096-62914559, default 62914559): Created a new partition 6 of type \u0026#39;Linux\u0026#39; and of size 4 GiB. Command (m for help): p Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xde6faae3 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 37750783 37748736 18G 83 Linux /dev/vda2 37750784 41945087 4194304 2G 82 Linux swap / Solaris /dev/vda3 41945088 44042239 2097152 1G 83 Linux /dev/vda4 44042240 62914559 18872320 9G 5 Extended /dev/vda5 44044288 54530047 10485760 5G 83 Linux /dev/vda6 54532096 62914559 8382464 4G 83 Linux Command (m for help): w The partition table has been altered. Failed to add partition 5 to system: Device or resource busy Failed to add partition 6 to system: Device or resource busy The kernel still uses the old partitions. The new table will be used at the next reboot. Syncing disks. [root@server1 ~]# fdisk -l /dev/vda Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xde6faae3 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 37750783 37748736 18G 83 Linux /dev/vda2 37750784 41945087 4194304 2G 82 Linux swap / Solaris /dev/vda3 41945088 44042239 2097152 1G 83 Linux /dev/vda4 44042240 62914559 18872320 9G 5 Extended /dev/vda5 44044288 54530047 10485760 5G 83 Linux /dev/vda6 54532096 62914559 8382464 4G 83 Linux [root@server1 ~]# cat /proc/partitions major minor #blocks name 253 0 31457280 vda 253 1 18874368 vda1 253 2 2097152 vda2 253 3 1048576 vda3 253 4 9436160 vda4 253 16 10485760 vdb 11 0 7377920 sr0 [root@server1 ~]# partprobe /dev/vda Error: Partition(s) 5, 6 on /dev/vda have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should reboot now before making further changes. [root@server1 ~]# reboot Since we\u0026rsquo;re getting an error using partprobe, we should reboot and not continue modifying or managing partitions.\nCreating GPT Partitions with gdisk # If a disk is configured with a GUID Partition Table or if the disk has a size that goes beyond 2TiB you need to manage partitions with the gdisk utility.\nNever use gdisk on a disk that has been formatted with fdisk and already contains fdisk partitions. gdisk will detect MBR is present and convert this to GPT after which your computer will likely not boot anymore.\n[root@server1 ~]# gdisk /dev/vda GPT fdisk (gdisk) version 1.0.3 Partition table scan: MBR: MBR only BSD: not present APM: not present GPT: not present *************************************************************** Found invalid GPT and valid MBR; converting MBR to GPT format in memory. THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by typing \u0026#39;q\u0026#39; if you don\u0026#39;t want to convert your MBR partitions to GPT format! *************************************************************** Warning! Secondary partition table overlaps the last partition by 33 blocks! You will need to delete this partition or resize it in another utility. Command (? for help): q In the following example, I\u0026rsquo;ll create a GPT layout on a new disk, /dev/vdb:\n[root@server1 ~]# gdisk /dev/vdb GPT fdisk (gdisk) version 1.0.3 Partition table scan: MBR: not present BSD: not present APM: not present GPT: not present Creating new GPT entries. Type p to print the current disk allocation:\nCommand (? for help): p Disk /dev/vdb: 20971520 sectors, 10.0 GiB Sector size (logical/physical): 512/512 bytes Disk identifier (GUID): 32689440-34F8-4D19-8FCC-0FA78AFFCAF5 Partition table holds up to 128 entries Main partition table begins at sector 2 and ends at sector 33 First usable sector is 34, last usable sector is 20971486 Partitions will be aligned on 2048-sector boundaries Total free space is 20971453 sectors (10.0 GiB) Number Start (sector) End (sector) Size Code Name Type n to create a new partition, accept the default partition number that is suggested. Accept the default suggested First sector. The last sector should be set at 1GiB.\nBy default the Linux partition type is selected. Accept or type l to show other parition types. Relevant partition types are as follows:\n8200 Linux Swap 8300 Linux File System 8e00 Linux LVM These are the same types as the ones used in MBR, except that two 0s are added to the ID.\nCommand (? for help): n Partition number (1-128, default 1): First sector (34-20971486, default = 2048) or {+-}size{KMGTP}: Last sector (2048-20971486, default = 20971486) or {+-}size{KMGTP}: +1G Current type is \u0026#39;Linux filesystem\u0026#39; Hex code or GUID (L to show codes, Enter = 8300): Changed type of partition to \u0026#39;Linux filesystem\u0026#39; Check the partition by printing the disk allocation:\nCommand (? for help): p Disk /dev/vdb: 20971520 sectors, 10.0 GiB Sector size (logical/physical): 512/512 bytes Disk identifier (GUID): 32689440-34F8-4D19-8FCC-0FA78AFFCAF5 Partition table holds up to 128 entries Main partition table begins at sector 2 and ends at sector 33 First usable sector is 34, last usable sector is 20971486 Partitions will be aligned on 2048-sector boundaries Total free space is 18874301 sectors (9.0 GiB) Number Start (sector) End (sector) Size Code Name 1 2048 2099199 1024.0 MiB 8300 Linux filesystem Type w to write your changes.\nCommand (? for help): w Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING PARTITIONS!! Do you want to proceed? (Y/N): y OK; writing new GUID partition table (GPT) to /dev/vdb. The operation has completed successfully. Again, if you get an error message indicating that the partition table is in use, type partprobe to update the kernel partition table and reboot if required.\n[root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 7G 0 rom vda 253:0 0 30G 0 disk ├─vda1 253:1 0 18G 0 part / ├─vda2 253:2 0 2G 0 part [SWAP] ├─vda3 253:3 0 1G 0 part ├─vda4 253:4 0 1K 0 part ├─vda5 253:5 0 5G 0 part └─vda6 253:6 0 4G 0 part vdb 253:16 0 10G 0 disk └─vdb1 253:17 0 1G 0 part [root@server1 ~]# cat /proc/partitions major minor #blocks name 253 0 31457280 vda 253 1 18874368 vda1 253 2 2097152 vda2 253 3 1048576 vda3 253 4 1 vda4 253 5 5242880 vda5 253 6 4191232 vda6 253 16 10485760 vdb 253 17 1048576 vdb1 11 0 7377920 sr0 Creating GPT Partitions with parted # The parted utility uses an interactive shell, it\u0026rsquo;s considered to be the default utility on RHEL 8 but it lacks advanced features.\nI\u0026rsquo;ve deleted the partion I created previously with gdisk and I\u0026rsquo;ll recreate it using parted:\n[root@server1 ~]# parted /dev/vdb GNU Parted 3.2 Using /dev/vdb Welcome to GNU Parted! Type \u0026#39;help\u0026#39; to view a list of commands. print the current disk allocation table:\n(parted) print Model: Virtio Block Device (virtblk) Disk /dev/vdb: 10.7GB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags Type mklabel and press Enter. You\u0026rsquo;ll be prompted for a disk label type, press the Tab key twice to see a list of available disk label types. Choose the gpt type and press Enter.\n(parted) mklabel New disk label type? aix amiga atari bsd dvh gpt loop mac msdos pc98 sun New disk label type? gpt Warning: The existing disk label on /dev/vdb will be destroyed and all data on this disk will be lost. Do you want to continue? Yes/No? yes Type mkpart, the utility prompts for a partition name. I\u0026rsquo;ve named by partition backup.\n(parted) mkpart Partition name? []? backup Notice you\u0026rsquo;re prompted for a file system type, it suggest we\u0026rsquo;re applying a file system here but this is not the case. Documentation suggest this setting isn\u0026rsquo;t used, you could accept the default, but it\u0026rsquo;s suggested to use Tab completion and choose a file system type comes close to what you\u0026rsquo;re going to use on the partition later when actually creating the file system.\nFile system type? [ext2]? affs0 affs3 affs6 amufs0 amufs3 apfs1 btrfs ext4 hfs hp-ufs linux-swap(new) linux-swap(v1) reiserfs xfs affs1 affs4 affs7 amufs1 amufs4 apfs2 ext2 fat16 hfs+ jfs linux-swap(old) nilfs2 sun-ufs affs2 affs5 amufs amufs2 amufs5 asfs ext3 fat32 hfsx linux-swap linux-swap(v0) ntfs swsusp File system type? [ext2]? xfs We can specify the start location as a number of blocks or an offset from the start of the device like 1MiB, not +1MiB. We\u0026rsquo;ll make the partition 5GiB in size, so we specify the 5Gib offset for the end value.\nStart? 1MiB End? 5GiB Type print to see the modifications and quit to quit the utility and commit changes.\n(parted) print Model: Virtio Block Device (virtblk) Disk /dev/vdb: 10.7GB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 1 1049kB 5369MB 5368MB xfs backup (parted) quit Information: You may need to update /etc/fstab. [root@server1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 7G 0 rom vda 253:0 0 30G 0 disk ├─vda1 253:1 0 18G 0 part / ├─vda2 253:2 0 2G 0 part [SWAP] ├─vda3 253:3 0 1G 0 part ├─vda4 253:4 0 1K 0 part ├─vda5 253:5 0 5G 0 part └─vda6 253:6 0 4G 0 part vdb 253:16 0 10G 0 disk └─vdb1 253:17 0 5G 0 part [root@server1 ~]# ","date":"29 January 2021","externalUrl":null,"permalink":"/managing-storage-creating-partitions/","section":"Blog","summary":"","title":"Managing Storage: Creating Partitions","type":"posts"},{"content":"","date":"29 January 2021","externalUrl":null,"permalink":"/tags/mbr/","section":"Tags","summary":"","title":"MBR","type":"tags"},{"content":"","date":"29 January 2021","externalUrl":null,"permalink":"/tags/parted/","section":"Tags","summary":"","title":"Parted","type":"tags"},{"content":"","date":"29 January 2021","externalUrl":null,"permalink":"/tags/partitions/","section":"Tags","summary":"","title":"Partitions","type":"tags"},{"content":" If a mass storage device is connected to a Linux computer, the Linux kernel tries to locate any partitions. So to use a hard drive we need to partition it. On RHEL 8 two different partitioning schemes are available: the Master Boot Record and GUID Partition Table. Linux typically has multiple partitions on one hard disk, this makes sense for different reasons:\nDistinguish different types of data. Mount options to enhance security or performance. Create backup strategies where only relevant portions of the OS are backed up. If one partition accidentally fills up completely other partitions are still usable and your system might not crash immediately. The MBR Partitioning Scheme # In the early 1980s the Master Boot Record partitioning scheme was invented to define hard disk layout. On BIOS-based systems the BIOS searches for an operating system on the bootable disk device where the first 512 bytes of the boot medium is read. The MBR is defined as these first 512 bytes which contains an operating system boot loader (446 bytes) and a partition table (64 bytes). The size used for the partition table, means no more than 4 partions can be created with a maximum size of 2 TiB per partition. The last 2 bytes (0x55, 0xaa) of the first 512 byte sector signifies the device is bootable. These two bytes are also call the \u0026ldquo;magic number\u0026rdquo;.\nTo go beyond the limit of 4 partitions, one partition could be created as an extended partition (as opposed to a primary partition) where multiple logical partitions can be created within to reach a total of 15 partitions addressable by the Linux kernel.\nThe GPT Partitioning Scheme # Current hard drives have become too big to be addressed by MBR partitions, the GUID Partition Table counters this. On computers that use the new Unified Extensible Firware Interface (UEFI) instead of BIOS, GPT partitions are the only way to address disks. Systems using BIOS can use GUID partitions and must do so if a disk bigger than 2 TiB needs to be addressed. Although not being used by UEFI devices, the MBR is present at the beginning of the disk, in block LBA0, for protective and compatibility purposes. Strictly speaking, the GPT starts up from the Partition Table Header.\nThe benefits of using GUID:\na 8 ZiB partition size (1024^4). Maximum 128 partitions. No distinguishment between primary, extended and logical partitions. Uses 128-bit global unique identifier (GUID) to identify partitions. A backup of the GUID partition table is created at the end of the disk (Secondary GPT Header), eliminating the single point of failure of MBR partition tables. Storage Measurement Units # Different measurement units can be used, we typically differentiate between binary (power of 2) and decimal (power of 10) measurement units. Decimal units use a multiple of 1,000 bytes while binary units use a multiple of 1,024 bytes.\nWhen speaking of partitions using binary units, the starting point of the partition is aligned to the exact sector specified by size and the ending point is aligned to the specified size minus 1 sector. If using decimal units, the starting and ending point is aligned within one half of the specified unit: for example, ±500KB when using the MB suffix.\nIn computers it makes sense to use binary units because that\u0026rsquo;s how computers address items, the binary unit has become the standard on current Linux distribution although some utilities may provide their output in decimal units.\nSymbol Name Value Symbol Name Value KB Kilobyte 1000^1 KiB Kibibyte 1024^1 MB Megabyte 1000^2 KiB Mebibyte 1024^2 GB Gigabyte 1000^3 KiB Gibibyte 1024^4 TB Terabyte 1000^4 KiB Tebibyte 1024^5 PB Petabyte 1000^5 KiB Pebibyte 1024^6 EB Exabyte 1000^6 KiB Exbibyte 1024^7 ZB Zettabyte 1000^7 KiB Zebibyte 1024^8 YB Yottabyte 1000^8 KiB Yobibyte 1024^9 ","date":"26 January 2021","externalUrl":null,"permalink":"/managing-storage-understanding-mbr-and-gpt-partitions/","section":"Blog","summary":"","title":"Managing Storage: Understanding MBR and GPT Partitions","type":"posts"},{"content":"","date":"18 December 2020","externalUrl":null,"permalink":"/tags/logging/","section":"Tags","summary":"","title":"Logging","type":"tags"},{"content":"","date":"18 December 2020","externalUrl":null,"permalink":"/tags/rhcsa/","section":"Tags","summary":"","title":"RHCSA","type":"tags"},{"content":" Understanding System Logging # Services on a Linux server write information to log files in different destinations using different approaches, and there are multiple ways to find relevant information in those log files.\nServices can write log information via:\nDirect Write - information is written directly to a log file by a service. rsyslogd - a service passes log information to the rsyslogd service that in turn takes care of managing centralized log files. journald - this service is tightly integrated with Systemd. journald receives all messages generated by Systemd Units. It also collects messages from the kernel, the boot procedure and services, then writes these message to an event journal which is stored in binary format. This journal can be queried with the journalctl command.\njournald is not persistent between reboots, so messages are also forwarded to rsyslogd which in turn writes the log information to different files in the /var/log directory and thus making them persistent.\nrsyslogd adds some services to journald, it allows you to configure remote logging and log servers, and makes logging persistent.\nWe also have the auditd service which offers an in-depth trace of what specific services, processes and users have been doing.\nWe the above in mind, we have three approaches to get more information on what has been happening on a machine:\nMonitor the files in /var/log written by rsyslogd or by services directly. Use the journalctl command. Use the systemctl status command. Persistent Log Files # As mentioned earlier, persistent log files are located in the /var/log directory. Most of the files are managed by rsyslogd but some of them are created directly by services. The most convenient way of reading these files would be by using a pager like less.\nThe below table provides an overview of some of the standard files that are created in this directory and what content you may expect.\nLog File Explanation /var/log/messages A generic log file where most messages are written to. /var/log/dmesg Contains kernel log messages. /var/log/secure Contains authentication-related messages. /var/log/boot.log Messages related to system startup /var/log/audit/audit.log Contains audit messages. SELinux writes to this file. /var/log/maillog Mail-related messages /var/log/samba The Samba service writes directly to this file, it\u0026rsquo;s not managed through rsyslogd. /var/log/sssd Messages written by the sssd service, which plays an important role in the authentication process. /var/log/cups Messages generated the the CUPS print service. /var/log/httpd Directory that contains log files written directly by the Apache web server. It might be useful to see what is happening in real time. The tail -f \u0026lt;logfile\u0026gt; command would allow you to live monitor a specific log file, you\u0026rsquo;ll see lines being added to the output of the command as system events happen.\nUsing logger # While services write information to log files by themselves or through rsyslogd, users can write messages to rsyslogd from the command line or a script using the logger command. Simply type logger followed by the message you want to write to the logs.\nYou can also specify the priority and facility to log to. For example, logger -p kern.err hello would write hello to the kernel facility using the error priority.\nConfiguring rsyslogd # The rsyslogd service makes sure that information that needs to be logged is written to the location where you would want to find it. This is configured inside the /etc/rsyslog.conf file that acts as the central location from where rsyslogd is configured. From this file the /etc/rsyslog.d/ directory is included, RPM packages can put specific configuration files inside this directory.\nIf you need to pass specific options to the rsyslogd service on startup you need to do that via the /etc/sysconfig/ryslog file. This file contains one line by default where you can specify startup parameters: SYSLOGD_OPTIONS=\u0026quot;\u0026quot; This variable is included in the Systemd configuration file that starts rsyslogd.\nUnderstanding rsyslogd Configuration Files # The rsyslogd.conf file contains different sections to specify what should be logged and where:\n### MODULES ###: rsyslogd is modular and you can expand features using modules.\n### GLOBAL DIRECTIVES ###: Specified global parameters like the default timestamp format or the location where files are being written to.\n### RULES ###: This is the most important part, it contains rules that specify which information should be logged to which destination.\n#### RULES #### # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don\u0026#39;t log private authentication messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages *.emerg :omusrmsg:* # Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log Facilities, Priorities and Destinations # The Rules section we discussed above uses facilities, priorities and destinations to specify what should be logged and where.\nA facility specifies a category of information. There is a fixed list of facilities that cannot be extended to ensure backwards compatibility. As a result some facilities exist that serve no purpose anymore and some services that became relevant at a later stage do not have their own facility. To circumvent that you can configure the service to use the generic daemon and local0 to local7 facilities. Facility Used by auth/authpriv Messages related to authentication cron Messages generated by the crond service daemon Generic facility to be used for nonspecified daemons kern Kernel messages lpr Messages generated through the legacy lpd print system mail Email related messages mark Special facility that can be used to periodically write a marker news Messages generated by the NNTP news system security Deprecated, same as auth/authpriv syslog Messages generated by the syslog system user Messages generated in user space uucp Messages generated by the legacy UUCP system local0-7 Messages generated by services configured to use one of the local0 to local7 facilities Priorities define the severity of the message that needs to be logged. When specifying a priority, all messages with that specific priority and higher will be logged.\nIf you need to configure logging where messages with different priorities are sent to different files you can specify the priority with a = sign: cron.=debug -/var/log/cron.debug This will write all cron message with only the debug priority to the /var/log/cron.debug log. Priority Description emerg/panic Message generated when the availability of the service is discontinued alert Used when the availability of the service is about to be discontinued crit A critical error has occurred error/err A noncritical error has occurred warning/warn Something is suboptimal, but no real error yet notice Messages about items that might become an issue later info Informational messages about normal service operation debug Debug messages that will give as much information as possible about service operation Destinations defines where the message should be written to. Typically these are files, but they can be devices or ryslog modules as well. If the destination file name starts with a hypen, e.g. -/var/log/maillog, the message will not be commited immediately to the file. Instead, it will be buffered to make writes more efficient. See man 5 rsyslog.conf\nRotating Log Files # When a certain treshold has been reached the old log file is closed and a new log file is opened by the logrotate utility which is started periodically by the crond service.\nWhen a log file is rotated, the old log file is typically copied to a new file that has the rotation date in it. By default, four old log files are kept on the system, files older than that period are removed from the system automatically.\nYou should back up log files or configure a centralized log server where logrotate keeps rotated messages for a significanlty longer period.\nThe default settings for log rotation are kep in the /etc/logrotate.conf file, but you can create configuration files inside the /etc/logrotate.d/ directory to be applied to specific log files.\n# see \u0026#34;man logrotate\u0026#34; for details # rotate log files weekly weekly # keep 4 weeks worth of backlogs rotate 4 # create new (empty) log files after rotating old ones create # use date as a suffix of the rotated file dateext # uncomment this if you want your log files compressed #compress # packages drop log rotation information into this directory include /etc/logrotate.d # system-specific logs may be also be configured here. Working with journald # The systemd-journald service stores log messages in a binary file, the journal, /run/log/journal. Since this file is in the /run directory, it is not persistent.\nWe can examine the journal using the journalctl command, the output is shown in a less pager and by default you\u0026rsquo;ll see the beginning of the journal. You can live monitor the journal using the journcalctl -f command.\njournalctl has many filtering options, you can find some of the most useful in the table below. You can also use tab completion to see specific options.\nCommand Description journalctl --no-pager Shows the content without using a pager journalctl -f Live monitor the journal as events happen journalctl _UID=1000 Show messages that have been logged for a specific user journalctl -p err Show messages with the error priority journalctl --since yesterday --until today Shows messages in a specific time frame. Accepts keywords like yesterday, today, tomorrow or you can specify a date/time in the YYYY-MM-DD hh:mm:ss format. journalctl _SYSTEMD_UNIT=sshd.service Shows information on the sshd Systemd Unit. journalctl --dmesg Shows kernel related messages journalctl -o verbose Shows detailed information on logged items The above options can all be combined, e.g. journalctl --since yesterday -p err _UID=1000 or journalctl _SYSTEMD_UNIT=sshd.service -o verbose. Persistently Storing the Systemd Journal # Storing the journal in a persistent way so that it survives reboots requires setting the Storage= parameter inside /etc/systemd/journal.conf:\nStorage=auto The journal will be written to disk if the /var/log/journal directory exists. Storage=volatile The journal will only be stored in /run/log/journal in a non persistent way. Storage=persistent The journal will be written to disk in the /var/log/journal directory, if this directory does not exist it will be created. Storage=none No data will be stored, but forwarding to other targets like the kernel log buffer or syslog will still work. If you manually created the /var/log/journal directory you need to set ownership and permissions, then either reboot or kill the systemd-journald service. Restarting the systemd-journald service is not enough:\n# chown root:systemd-journal /var/log/journal # chmod 2755 /var/log/journal # killall -USR1 systemd-journald The journal has built-in log rotation, so even if the journal is stored persistently the data is not kept forever. The journal is limited to a maximum size of 10% of the file system it\u0026rsquo;s on and it will stop growing when less than 15% of file system is free. When this happens the oldest messages from the journal are dropped to make room for new messages. This can be changed in the /etc/systemd/journal.conf file.\n","date":"18 December 2020","externalUrl":null,"permalink":"/understanding-and-configuring-system-logging/","section":"Blog","summary":"","title":"Understanding and Configuring System Logging","type":"posts"},{"content":"","date":"23 November 2020","externalUrl":null,"permalink":"/tags/anacron/","section":"Tags","summary":"","title":"Anacron","type":"tags"},{"content":"","date":"23 November 2020","externalUrl":null,"permalink":"/tags/at/","section":"Tags","summary":"","title":"At","type":"tags"},{"content":"","date":"23 November 2020","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"23 November 2020","externalUrl":null,"permalink":"/tags/cron/","section":"Tags","summary":"","title":"Cron","type":"tags"},{"content":"","date":"23 November 2020","externalUrl":null,"permalink":"/categories/red-hat-enterprise-linux/","section":"Categories","summary":"","title":"Red Hat Enterprise Linux","type":"categories"},{"content":" Configuring Cron to Automate Recurring Tasks. # On Linux, the cron service is used to run processes automatically at specific times as a way to automate tasks that have to occur regularly.\nThe cron service consists of two major components:\nThe cron daemon checks every minute to see if there are any jobs to run. The cron configuration files work together to provide the right information to the right service at the right time. Managing the crond Service # Because some system tasks run through the crond service, this service is started by default. The crond service itself doesn\u0026rsquo;t need much management, it does not need to be reloaded or activated, the crond daemon wakes up every minute to see if anything needs to be run. You can monitor the current status of the service using the systemctl status crond -l command:\n[root@server1 ~]# systemctl status crond -l ● crond.service - Command Scheduler Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2020-07-04 14:22:31 +04; 3 days ago Main PID: 9134 (crond) Tasks: 1 (limit: 49648) Memory: 3.4M CGroup: /system.slice/crond.service └─9134 /usr/sbin/crond -n Jul 07 12:01:01 server1 CROND[56917]: (root) CMD (run-parts /etc/cron.hourly) Jul 07 13:01:01 server1 CROND[57001]: (root) CMD (run-parts /etc/cron.hourly) Jul 07 14:01:01 server1 CROND[57303]: (root) CMD (run-parts /etc/cron.hourly) Jul 07 15:01:01 server1 CROND[57551]: (root) CMD (run-parts /etc/cron.hourly) Jul 07 16:01:01 server1 CROND[57582]: (root) CMD (run-parts /etc/cron.hourly) Jul 07 17:01:01 server1 CROND[57621]: (root) CMD (run-parts /etc/cron.hourly) Jul 07 18:01:01 server1 CROND[57715]: (root) CMD (run-parts /etc/cron.hourly) Jul 07 19:01:01 server1 CROND[57807]: (root) CMD (run-parts /etc/cron.hourly) Jul 07 19:01:01 server1 run-parts[57816]: (/etc/cron.hourly) finished 0anacron Jul 07 20:01:01 server1 CROND[57845]: (root) CMD (run-parts /etc/cron.hourly) The systemctl command uses the journald service to find out what is happening with the crond service.\nCron Timing # A time string is used to specify when exactly a specific job should be run. The following cron time and date fields can be used for this time string:\nField Values minute 0 - 59 hour 0 - 23 day of month 1 - 31 month 1 - 12 day of week 0 - 7 (Sunday is 0 or 7) In any of the above fields you can use an * as a wildcard to refer to any value, ranges of numbers, lists and patterns are also allowed:\n* 11 * * * - Every minute between 11:00 and 11:59 0 11 * * 1-5 - Every weekday at 11:00 0 7-18 * * 1-5 - Every weekday between 7a.m. and 6p.m. at the top of the hour 0 */2 2 12 5 - Every 2 hours at the top of the hour, on the 2nd of December and every Friday in December. # For details see man 4 crontabs # Example of job definition: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | .---------- day of month (1 - 31) # | | | .------- month (1 - 12) OR jan,feb,mar,apr ... # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat # | | | | | # * * * * * user-name command to be executed Cron Configuration Files # The main configuration file for cron is /etc/crontab, but you will not change this file directly. Different Cron configuration files are used:\nCron files in /etc/cron.d Scripts in /etc/cron.hourly, cron.daily, cron.weekly and cron.monthly User-specific files created with crontab -e Cron jobs can be created for specific users by either logging in as that user and executing crontab -e or executing crontab -e -u USERNAME as root. crontab -e opens the vi editor and creates a temporary file. After saving your changes, the temporary file is moved to /var/spool/cron where a file is created for each user. These files should not be edited directly.\nYou can also add Cron jobs that are not tied to a specific user account, these will be executed by default as root unless you specify otherwise. To do so you add a file where the content meets the syntax of a typical cron job inside the /etc/cron.d directory, the file name does not matter.\nThe last way to schedule Cron jobs is through the following directories:\n/etc/cron.hourly /etc/cron.daily /etc/cron.weekly /etc/cron.monthly Scripts added to these directories should not contain any information on when it should be executed. You would only add scripts there if the exact time of execution does not really matter, the only thing that would matter is if the job needs to be launched once an hour, day, week or month.\nThe Purpose of Anacron # Anacron is the service that takes care of starting the hourly, daily, weekly and monthly jobs regardless of the exact time. Anacron uses the /etc/anacrontab file for this:\n[root@server1 cron.d]# cat /etc/anacrontab # /etc/anacrontab: configuration file for anacron # See anacron(8) and anacrontab(5) for details. SHELL=/bin/sh PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root # the maximal random delay added to the base delay of the jobs RANDOM_DELAY=45 # the jobs will be started during the following hours only START_HOURS_RANGE=3-22 #period in days delay in minutes job-identifier command 1\t5\tcron.daily\tnice run-parts /etc/cron.daily 7\t25\tcron.weekly\tnice run-parts /etc/cron.weekly @monthly 45\tcron.monthly\tnice run-parts /etc/cron.monthly From the above we can see that Anacron will only run jobs between 3a.m. and 10p.m.\nThe period in days specifies the job execution frequency, delay in minutes specifies how long Anacron waits before executing the job. Then we have the job identifier cron.daily and the command that\u0026rsquo;s being executed (nice run-parts /etc/cron.daily)\nThe need to configure jobs through Anacron is taken away by the /etc/cron.hourly, cron.daily, cron.weekly and cron.monthly directories.\nNote that there is no single command that would show all currently scheduled jobs. The crontab -l command does list cron jobs, but only for the current user.\nCron Security # By default, all users are allowed to create Cron jobs.\nTo limit which user is allowed or not allowed to create Cron jobs we use the /etc/cron.allow and /etc/cron.deny files:\nIf the cron.allow file exists, a user must be listed in the file to be allowed to use Cron. If the cron.deny file exists, a user must not be listed in the file to be allowed to use cron. Both files should not exist at the same time.\nIf neither file exists, only root can use Cron.\nConfiguring At to Schedule Future Tasks # The atd service is used for jobs that need to be executed only once and are thus not recurring. You can use the at command followed by the time the job needs the be executed, either a specific time as in at 14:00 or a time indication like at noon or at teatime.\nAfter typing the at command a shell opens where you can type several commands that will be executed on at the time you specified, press CTRL-D to leave the shell.\nThe atq command lists an overview of all currently scheduled jobs. You can remove a specific job using the atrm command followed by the job number.\nA load value can be specified when starting the atd service using the -l option.\ne.g. atd -l 3.0 will make sure no job is started when the system load is higher than 3.0\n[student@server1 ~]$ at noon warning: commands will be executed using /bin/sh at\u0026gt; echo \u0026#34;It\u0026#39;s noon\u0026#34; at\u0026gt; \u0026lt;EOT\u0026gt; job 41 at Wed Jul 08 12:00:00 2020 [student@server1 ~]$ atq 41\tWed Jul 08 12:00:00 2020 a student [student@server1 ~]$ atrm 41 [student@server1 ~]$ atq [student@server1 ~]$ Instead of specifying commands to be executed interactively from the prompt, we can instruct at to execute an existing script or program simply by passing it as an argument to the -f flag (or by using input redirection):\n[student@server1 ~]$ at now + 1 minute -f script.sh warning: commands will be executed using /bin/sh job 42 at Wed Jul 08 12:02:00 2020 [student@server1 ~]$ [student@server1 ~]$ at now + 1 minute \u0026lt; at_jobs.txt warning: commands will be executed using /bin/sh job 43 at Wed Jul 08 12:05:00 2020 [student@server1 ~]$ ","date":"23 November 2020","externalUrl":null,"permalink":"/scheduling-tasks-with-cron-at-anacron/","section":"Blog","summary":"","title":"Scheduling Tasks with Cron, At and Anacron","type":"posts"},{"content":"","date":"23 November 2020","externalUrl":null,"permalink":"/tags/tasks/","section":"Tags","summary":"","title":"Tasks","type":"tags"},{"content":" Systemd is considered to be the future standard init system by all mainstream Linux distributions. The Systemd System and Service Manager uses \u0026ldquo;units\u0026rdquo; as an abstraction for parts of the system to be managed: typically services, sockets, mounts and targets. It provides a uniform interface for managing units.\nThe systemctl -t help command displays a list of available units that can be managed with Systemd.\n[student@server1 ~]$ systemctl -t help Available unit types: service mount swap socket target device automount timer path slice scope Each unit type can be recognized by the file extension. A service unit will end on .service while a target unit will end on .target.\nUnit files are located in three different locations:\n/usr/lib/systemd/system contains default unit files that have been installed by RPM packages. These should not be edited since package updates can overwrite your changes.\n/etc/systemd/system contains custom unit files written by a system administrator or generated by the systemctl edit command.\n/run/systemd/system contains unit files that have been generated automatically by the system during runtime.\nThe above locations have a specific priority if a unit file happens to exist in more than one location. Units in the /run directory have the highest priority, followed by custom unit files in the /etc/systemd/system directory and lastly the files inside /usr/lib/systemd/system.\nSystemd Service Units # The most important unit type is the service unit. You can start any type of process by using a service unit, including daemon processes and regular commands.\nLet\u0026rsquo;s take a look at the service unit file for the Very Secure Ftp Daemon:\n[root@server1 ~]# systemctl cat vsftpd # /usr/lib/systemd/system/vsftpd.service [Unit] Description=Vsftpd ftp daemon After=network.target [Service] Type=forking ExecStart=/usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf [Install] WantedBy=multi-user.target We can identify three sections in this service unit file:\n[Unit] This sections describes the unit and dependencies (more on that later). It contains the important After statement and optional Before statement. These statements define dependencies between different units and how they relate from the perspective of this unit. In the above example, the After statement indicates that this unit should be started after the network.target unit.\n[Service] describes how to start and stop the unit. Usually you will see the ExecStart statement to indicate how to start the unit and the ExecStop statement for how to stop the unit. The Type statement is used to specify how the process should start. The forking type is commonly used by daemon processes.\n[Install] indicates in which target unit this service has to start. We\u0026rsquo;ll cover more on target units later.\nFor more detailed information on the above, see man 5 systemd.service.\nSystemd Mount Units # A mount unit specifies how a file system can be mounted on a specific directory. Configuring mount points through /etc/fstab is the preferred approach, any mount points configured in the /etc/fstab file will be converted to a Systemd mount unit at boot time.\n[root@server1 ~]# systemctl cat tmp.mount [Unit] Description=Temporary Directory (/tmp) Documentation=https://systemd.io/TEMPORARY_DIRECTORIES Documentation=man:file-hierarchy(7) Documentation=https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems ConditionPathIsSymbolicLink=!/tmp DefaultDependencies=no Conflicts=umount.target Before=local-fs.target umount.target After=swap.target [Mount] What=tmpfs Where=/tmp Type=tmpfs Options=mode=1777,strictatime,nosuid,nodev\u0026lt;/code\u0026gt;\u0026lt;/pre\u0026gt; We typically see the following options inside the [Mount] section:\nWhat takes an absolute path of a file or device node. This is a mandatory option.\nWhere defines the absolute path for the mount point, which cannot be a symbolic link. If it does not exist at the time of mounting, it is created as a directory. This is a mandatory option as well and the string must reflect the unit filename.\nType defines the file system type. This is optional.\nSee man 5 systemd.mount for more details.\nSystemd Socket Units # A socket creates a method for applications to communicate either by means of a file or a TCP/UDP port on which Systemd will be listening for incoming connections.\nThis means a specific service doesn\u0026rsquo;t have to be running continuously, if Systemd detects an incoming connection on the socket it can start the associated service (on-demand starting). With that in mind, each socket unit must have a corresponding service unit file.\n[root@server1 ~]# systemctl cat cockpit.socket # /usr/lib/systemd/system/cockpit.socket [Unit] Description=Cockpit Web Service Socket Documentation=man:cockpit-ws(8) Wants=cockpit-motd.service [Socket] ListenStream=9090 ExecStartPost=-/usr/share/cockpit/motd/update-motd \u0026#39;\u0026#39; localhost ExecStartPost=-/bin/ln -snf active.motd /run/cockpit/motd ExecStopPost=-/bin/ln -snf /usr/share/cockpit/motd/inactive.motd /run/cockpit/motd [Install] WantedBy=sockets.target The important option in the above example is ListenStream. This options specifies the address and/or TCP port number to listen on. It can be written as e.g. 192.168.0.1:9090 or 9090 or /path/to/file.socket or [ipv6]:portnumber\nFor UDP ports you would use ListenDatagram instead.\nSee man 5 systemd.socket for more information.\nSystemd Target Units # A target unit makes it possible to load unit files in a specific order to define the state the machine should be started in, very similar to the runlevels used in other init systems. A target unit is basically a group of units, they don\u0026rsquo;t have additional functionality on top of the units they group.\n[root@server1 ~]# systemctl cat multi-user.target # /usr/lib/systemd/system/multi-user.target [Unit] Description=Multi-User System Documentation=man:systemd.special(7) Requires=basic.target Conflicts=rescue.service rescue.target After=basic.target rescue.service rescue.target AllowIsolate=yes The target unit has definitions on what it requires and what other targets it cannot coexist with (Conflicts). It defines load ordering by using the After option.\nWhen using the systemctl enable command on a unit to automatically start it at boot time, the [Install] section of that unit determines to what target unit it should be added. Behind the scenes, the systemctl enable command creates a symbolic link in the target directory (defined by [Install]) inside /etc/systemd/system.\nFor example, systemctl enable vsftpd adds a symbolic link in /etc/systemd/system/multi-user.target/wants/vsftpd.service which points to /usr/lib/systemd/system/vsftpd.service ensuring the vsftpd.service unit will be started automatically with the Multi-User System target. This symbolic link is called a want, it defines what the target wants to start when it is processed.\n[root@server1 ~]# systemctl enable vsftpd Created symlink /etc/systemd/system/multi-user.target.wants/vsftpd.service → /usr/lib/systemd/system/vsftpd.service. Using Systemd to Manage Units # You should\u0026rsquo;ve noticed we can use the systemctl command to manage Systemd units. systemctl start/stop will start/stop a unit, while systemctl enable/disable will either start the unit at boot or prevent the unit from starting at boot.\nThe systemctl status command shows us runtime status information on the unit as well as the most recent journal log data:\n[root@server1 ~]# systemctl status vsftpd ● vsftpd.service - Vsftpd ftp daemon Loaded: loaded (/usr/lib/systemd/system/vsftpd.service; disabled; vendor preset: disabled) Active: active (running) since Thu 2020-06-25 14:27:13 +04; 1s ago Process: 66451 ExecStart=/usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf (code=exited, status=0/SUCCESS) Main PID: 66452 (vsftpd) Tasks: 1 (limit: 28430) Memory: 636.0K CPU: 4ms CGroup: /system.slice/vsftpd.service └─66452 /usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf Jun 25 14:27:13 DELLG5 systemd\u0026amp;#91;1]: Starting Vsftpd ftp daemon... Jun 25 14:27:13 DELLG5 systemd\u0026amp;#91;1]: Started Vsftpd ftp daemon. The second line shows us the unit has been loaded into memory, but is not enabled at boot. The status of Loaded can also be error, not-found, bad-setting, masked or static.\nThe Active line shows the current state:\nrunning: The unit is running with active processes exited: A one time run was successfully completed. waiting: The unit is running and waiting for events. inactive (dead): The unit is not running In addition to the systemctl status command for a specific unit, the following commands can help you get a bigger picture of unit statuses:\nCommand Description systemctl –type=service Show service units systemctl list-units –type-service idem systemctl list-units –type=service –all Show active and inactive service units systemctl –failed –type=service Show failed services Managing Systemd Dependencies # Unit files can have dependencies, a unit file may Want, Require or Requisite one or more other units After or Before it can run. These keywords can be used in the [Unit] section of a unit file.\nRequires: If this unit loads, then units specified here will also load. Deactivating either unit will result in both units being deactivated. Requisite: If the unit listed here is not already loaded, then the unit will fail. Wants: The unit will try to load units specified here, but it will not fail if any of the specified units do fail. Before: The unit will start before the unit listed here. After: The unit will start after the unit listed here. You can request a list of unit dependencies using the systemctl list-dependencies command. Adding a unit to the command will show dependencies for that specific unit, and adding the - -reverse flag will show what units are dependent on that specific unit.\n[root@server1 ~]# systemctl list-dependencies vsftpd vsftpd.service ● ├─system.slice ● └─sysinit.target ● ├─dev-hugepages.mount ● ├─dev-mqueue.mount ● ├─dmraid-activation.service .... [oot@server1 ~]# systemctl list-dependencies multi-user.target multi-user.target ● ├─abrt-journal-core.service ● ├─abrt-oops.service .... [root@server1 ~]# systemctl list-dependencies --reverse multi-user.target multi-user.target ● └─graphical.target Managing Unit Options # There\u0026rsquo;s a huge amount of options available for each unit file, to see what options (and their default values) are available for a specific unit use the systemctl show command.\nChanges to unit files need to be written to the /etc/systemd/system directory which is where custom unit files are located. The recommended way is to do so by using the systemctl edit command on a unit, this will create an override file for the unit and store that in the correct location. All settings in this file overwrite existing settings in /usr/lib/systemd/system.\nBy default, systemctl edit will use the nano editor, but you can change the default file editor by including the following in ~/.bash_profile or /etc/bash_profile to change this system wide:\nexport SYSTEMD_EDITOR=\u0026quot;/bin/vim\u0026quot;\nPractical Example # Let\u0026rsquo;s have a look at a practical example where we will install the httpd and vsftpd services, enable them at boot time and have them auto restart after 5 seconds should they fail. When the httpd service is started, the vsftpd service should be started as well, but the httpd service should not fail if for some reason vsftpd can not be started.\n[root@server1 ~]# dnf install httpd vsftpd -y Last metadata expiration check: 0:02:49 ago on Fri 26 Jun 2020 12:36:40 +04. Dependencies resolved. ..... [root@server1 ~]# systemctl enable httpd \u0026amp;\u0026amp; systemctl enable vsftpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. Created symlink /etc/systemd/system/multi-user.target.wants/vsftpd.service → /usr/lib/systemd/system/vsftpd.service. [root@server1 ~]# systemctl start httpd \u0026amp;\u0026amp; systemctl start vsftpd [root@server1 ~]# Now that we have both services installed, enabled at boot time and running, let\u0026rsquo;s make sure they automatically restart in case they fail:\n[root@server1 ~]# systemctl edit httpd [root@server1 ~]# systemctl cat httpd # /etc/systemd/system/httpd.service.d/override.conf [Service] Restart=on-failure RestartSec=5s [root@server1 ~]# systemctl daemon-reload Apply the same configuration for the vsftpd service and test it:\nCheck for the Main PID of the service using the systemctl status command and send a SIGKILL signal. When checking the status again you should see Active: activating (auto-restart) and a few seconds later the service will be running again.\n[root@server1 ~]# systemctl status httpd ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/httpd.service.d └─override.conf Active: active (running) since Fri 2020-06-26 12:42:04 +04; 9min ago Docs: man:httpd.service(8) Main PID: 32234 (httpd) [root@server1 ~]# kill -9 32234 [root@server1 ~]# systemctl status httpd ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/httpd.service.d └─override.conf Active: activating (auto-restart) (Result: signal) since Fri 2020-06-26 12:51:33 +04; 2s ago Docs: man:httpd.service(8) Process: 32234 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=killed, signal=KILL) Now, let\u0026rsquo;s make sure that the vsftpd service is automatically started when httpd is started:\n[root@server1 ~]# systemctl edit httpd [root@server1 ~]# systemctl cat httpd # /etc/systemd/system/httpd.service.d/override.conf [Service] Restart=on-failure RestartSec=5s [Unit] Wants=vsftpd.service [root@server1 ~]# systemctl daemon-reload Stop both services then only start the httpd service again:\n[root@server1 ~]# systemctl stop httpd \u0026amp;\u0026amp; systemctl stop vsftpd [root@server1 ~]# systemctl start httpd [root@server1 ~]# systemctl status vsftpd ● vsftpd.service - Vsftpd ftp daemon Loaded: loaded (/usr/lib/systemd/system/vsftpd.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/vsftpd.service.d └─override.conf Active: active (running) since Fri 2020-06-26 13:21:15 +04; 4s ago Summary # We learned about different unit types in Systemd, the three sections of a Systemd unit file, understanding Target Units and managing units with the systemctl command.\n","date":"18 October 2020","externalUrl":null,"permalink":"/an-introduction-to-systemd-and-systemd-unit-files/","section":"Blog","summary":"","title":"An Introduction to Systemd and Systemd Unit files","type":"posts"},{"content":"","date":"18 October 2020","externalUrl":null,"permalink":"/tags/red-hat/","section":"Tags","summary":"","title":"Red Hat","type":"tags"},{"content":"","date":"18 October 2020","externalUrl":null,"permalink":"/tags/service-manager/","section":"Tags","summary":"","title":"Service Manager","type":"tags"},{"content":"","date":"18 October 2020","externalUrl":null,"permalink":"/tags/units/","section":"Tags","summary":"","title":"Units","type":"tags"},{"content":" On Linux, we can distinct between three major process types: Shell jobs, Daemons and Kernel threads.\nShell Jobs are commands started from the command line and are also referred to as interactive processes. Daemons are processes that provides services. They usually start when a computer is booted. Kernel threads are part of the Linux kernel. You can not manage them using the common tools discussed in this post. Managing Shell Jobs # Running Jobs in the Foreground and Background # When you type a command, a shell job is started automatically in the foreground occupying the terminal until it finishes. This makes sense for commands that take little time to complete or commands that require user interaction.\nIf you know a command will take a significant amount to complete and it doesn\u0026rsquo;t require user interaction, you can start it in the background by appending an \u0026amp; to the command, e.g. dnf update -y \u0026amp;\nYou can move the last job that was started in the background to the foreground using the fg command. If multiple jobs are running, append the job ID, shown by the jobs command, to the fg command.\nIt can happen you already executed a command and want to move it to the background. In this case use CTRL+Z to temporarily stop the job and execute the bg command to move it to the background. This does not remove the job from memory, it just pauses the job so that it can be managed. You can use CTRL+C to stop the current job and remove it from memory.\n[student@server1 ~]$ dd if=/dev/zero of=/dev/null ^Z [1]+ Stopped dd if=/dev/zero of=/dev/null [student@server1 ~]$ bg [1]+ dd if=/dev/zero of=/dev/null \u0026amp; [student@server ~]# jobs [1]+ Running dd if=/dev/zero of=/dev/null \u0026amp; [student@server1 ~]# fg 1 dd if=/dev/zero of=/dev/null ^C 9744376+0 records in 9744375+0 records out 4989120000 bytes (5.0 GB, 4.6 GiB) copied, 20.619 s, 242 MB/s CTRL+C will terminate the job immediately without closing properly, which could result in data loss. An alternative key sequence you can use is CTRL-D, which sends the End Of File (EOF) character to the current job. The job will stop waiting for further input, complete what it was doing and terminate in a proper way.\nCommand Use \u0026amp; (at the end of a command line) Start the command in the background. CTRL+Z Stop the current job temporarily so it can be managed. CTRL+D Send the EOF character to indicate it should stop waiting for input and close properly. CTRL+C Cancel the current job and remove it from memory. bg Continues the job that has been temporarily stopped with CTRL+Z in the background. fg Brings back to the foreground the last command that was moved to the background. jobs Shows which jobs are currently running in the background. Displays job ID\u0026rsquo;s that can be used as an argument to the bg and fg commands. Parent-Child Relations # When a process is started from a shell, it becomes a child process of that shell. The parent is needed to manage the child, all processes started from a shell are terminated when that shell is stopped.\nProcesses started in the background will not be killed when the parent shell from which they are started is killed. If the parent is killed, the child process becomes a child of systemd:\n[student@server1 ~]$ dd if=/dev/zero of=/dev/null \u0026amp; [1] 3471 [student@server1 ~]$ exit Open a new Terminal and check the running processes using ps fax:\n2293 ? Ss 0:00 /usr/lib/systemd/systemd --user 3540 ? R 0:32 \\_ dd if=/dev/zero of=/dev/null Using Common Command-Line Tools for Process Management # Understanding Processes and Threads # One process can start several worker threads. Threads can be handled by the different CPUs or CPU cores available on the machine. You can not manage individual threads, the programmer of the multithreaded application needs to define how threads relate to one another.\nAs such, kernel threads can not be managed. You can not adjust priority and neither can you kill them except by taking the entire machine down. It\u0026rsquo;s easy to recognize kernel threads, they have a name that is between square brackets:\n[root@server1 ~]# ps aux | head USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.1 0.7 179196 13988 ? Ss 16:09 0:02 /usr/lib/systemd/systemd --switched-root --system --deserialize 18 root 2 0.0 0.0 0 0 ? S 16:09 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? I 16:09 0:00 [rcu_gp] root 4 0.0 0.0 0 0 ? I 16:09 0:00 [rcu_par_gp] root 6 0.0 0.0 0 0 ? I 16:09 0:00 [kworker/0:0H-kblockd] root 8 0.0 0.0 0 0 ? I 16:09 0:00 [mm_percpu_wq] root 9 0.0 0.0 0 0 ? S 16:09 0:00 [ksoftirqd/0] root 10 0.0 0.0 0 0 ? I 16:09 0:00 [rcu_sched] root 11 0.0 0.0 0 0 ? S 16:09 0:00 [migration/0] Using ps to Get Process Information # The most common command to get an overview of currently running processes is ps. Without arguments, the ps command shows only the processes started by the current user.\nThere are different options to display different process properties:\nps aux displays a short summary of the active processes. ps -ef shows the name of the process, but also the exact command it was started with. ps fax shows hierarchical relationships between parent and child processes. Note that some options to the ps command don\u0026rsquo;t have to start with a hyphen.\n[student@server1 ~]$ dd if=/dev/zero of=/dev/null \u0026amp; [1] 4303 [student@server1 ~]$ ps | grep 4303 4303 pts/0 00:00:08 dd [student@server1 ~]$ ps aux | grep 4303 student 4303 96.5 0.0 7324 952 pts/0 R 16:50 0:17 dd if=/dev/zero of=/dev/null student 4319 0.0 0.0 12108 976 pts/0 R+ 16:51 0:00 grep --color=auto 4303 [student@server1 ~]$ ps -ef | grep 4303 student 4303 4271 99 16:50 pts/0 00:00:23 dd if=/dev/zero of=/dev/null student 4327 4271 0 16:51 pts/0 00:00:00 grep --color=auto 4303 [student@server1 ~]$ ps fax | grep -b5 4303 16305- 2727 ? Ssl 0:00 \\_ /usr/libexec/evolution-addressbook-factory 16379- 2764 ? Sl 0:00 | \\_ /usr/libexec/evolution-addressbook-factory-subprocess --factory all --bus-name org.gnome.evolution.dataserver.Subprocess.Backend.AddressBookx2727x2 --own-path /org/gnome/evolution/dataserver/Subprocess/Backend/AddressBook/2727/2 16643- 3811 ? Ssl 0:00 \\_ /usr/libexec/gvfsd-metadata 16702- 4192 ? Ssl 0:00 \\_ /usr/libexec/gnome-terminal-server 16768- 4271 pts/0 Ss 0:00 | \\_ bash 16808: 4303 pts/0 R 0:39 | \\_ dd if=/dev/zero of=/dev/null 16876- 4342 pts/0 R+ 0:00 | \\_ ps fax 16922: 4343 pts/0 S+ 0:00 | \\_ grep --color=auto -b5 4303 16988- 4202 ? Sl 0:00 \\_ /usr/libexec/gnome-control-center-search-provider 17069- 2316 ? Sl 0:00 /usr/bin/gnome-keyring-daemon --daemonize --login 17146- 2446 tty2 Sl 0:00 /usr/libexec/ibus-x11 --kill-daemon 17209- 2529 ? Ss 0:04 /usr/libexec/sssd/sssd_kcm --uid 0 --gid 0 --logger=files 17294- 2646 ? Ssl 0:00 /usr/bin/spice-vdagent [student@server1 ~]$ kill 4303 [1]+ Terminated dd if=/dev/zero of=/dev/null [student@server1 ~]$ You may have noticed I\u0026rsquo;ve used an important piece of information in my ps |grep commands above: the PID or process ID. An alternative way would be to use the pgrep command to get a list of all PIDs that have a name containing the string dd.\nAdjusting Process Priority with nice and renice # Processes are started with a specific priority. All regular processes are equal and started with the same priority: 20.\nIn some cases it\u0026rsquo;s useful to change the default priority and to do that we can use nice and renice. Use nice to start a process with an adjusted priority and renice to change the priority of an already running process.\nYou can select values ranging from -20 to 19. The default niceness of a process is set to 0, which results in the priority value of 20 (the lowest priority available). A negative niceness increases the process priority while a positive niceness decreases the priority. Best practice would be to use increments of 5 to see how that impacts the process.\nKernel threads are started as real-time processes, you will never be able to block out kernel threads from CPU time by increasing the priority of a user process.\nSingle-threaded processes running with the highest priority (niceness -20), can never get beyond the boundaries of the CPU its running on.\nRegular users can only decrease priority of a running process, you need to be root to give processes increased priority.\n[student@server1 ~]$ nice -n 5 dd if=/dev/zero of=/dev/null \u0026amp; [1] 4666 [student@server1 ~]$ renice -n 0 -p 4666 renice: failed to set priority for 4666 (process ID): Permission denied [student@server1 ~]$ renice -n 10 -p 4666 4666 (process ID) old priority 5, new priority 10 [student@server1 ~]$ kill 4666 [1]+ Terminated nice -n 5 dd if=/dev/zero of=/dev/null [student@server1 ~]$ Notice the line that says old priority 5, new priority 10.\nThis is actually misleading, it should say niceness instead:\nThe default process priority is 20 (which is a niceness of 0), so setting the niceness to 5 will lower the priority to 25. You can check that with the top command which I will come back to later on:\nKill Signals with kill, killall and pkill # Remember that process have a parent-child relationship, the parent is responsible for the child process it created and killing a parent process will make all child process become children of the systemd process.\nThe Linux kernel allows many signals to be sent to process. We\u0026rsquo;ll discuss 3 major signals that work for all processes:\nSIGTERM (15): Ask a process to stop. SIGKILL (9): Force a process to stop. SIGHUP (1): Hang up a process. The process will reread its configuration files. This comes in handy after making changes to a process configuration file. To send a signal to a process, we use the kill command followed by the PID of the process. By default this will send the SIGTERM signal, the process will stop gracefully and close all open files.\nA process can however choose to ignore the SIGTERM signal, in that case we can force stop the process by sending the SIGKILL signal: kill -9 \u0026lt;pid\u0026gt;\nIn general it\u0026rsquo;s a bad idea to use the SIGKILL signal since you risk losing data and your system may become unstable if other processes depend on the killed process.\nAs an alternative to the kill command we have the pkill and killall commands. pkill takes the process name as an argument instead of the PID and killall will kill all processes using the same name simultaneously.\nUsing top to Manage Processes # top gives an overview of the most active processes currently running and allows you to do all previously discussed process management tasks.\u0026lt;\nThe 8th column (S) shows the process state:\nState Meaning Running (R) The process is currently running and using CPU time. Sleeping (S) The process is waiting for an event to complete. Uninterruptible sleep (D) The process is in a sleep state that can not be stopped, usually while waiting for I/O. Stopped (T) The process has been stopped. This typically happens to a shell job using the CTRL+Z sequence. Zombie (Z) The process was stopped but could not be removed by the parent, putting it into an unmanageable state. From within top use the r keyboard button to adjust the priority/niceness of a process and the k keyboard button to send kill signals to a process.\nThe load average is another important piece of information you can get with top. The load average is expressed as the number of processes that are in a running state (R) or blocking state (D) and is shown for the last 1, 5 and 15 minutes. You can get the same load average information using the uptime command:\n[student@server1 ~]$ uptime 17:50:22 up 1:41, 1 user, load average: 0.01, 0.08, 0.27 As a rule of thumb, the load average should not be higher than the number of CPUs or CPU cores on the system. If the load average over a longer period of time is higher than the number of CPUs there may be a performance issue. You can check the number of CPUs and/or cores using the lscpu command:\n[student@server1 ~]$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 2 On-line CPU(s) list: 0,1 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 2\u0026lt;/code\u0026gt;\u0026lt;/pre\u0026gt; In the above example I have 2 CPUs, so the load average on my system over a longer period of time should not be above 2.\nSummary # We learned how to create and manage background jobs, lookup specific processes, terminating processes and changing priorities.\n","date":"3 October 2020","externalUrl":null,"permalink":"/managing-shell-jobs-and-processes-on-rhel8/","section":"Blog","summary":"","title":"Managing Shell Jobs and Processes on RHEL8","type":"posts"},{"content":"","date":"3 October 2020","externalUrl":null,"permalink":"/tags/processes/","section":"Tags","summary":"","title":"Processes","type":"tags"},{"content":"","date":"4 July 2020","externalUrl":null,"permalink":"/tags/appstream/","section":"Tags","summary":"","title":"AppStream","type":"tags"},{"content":"","date":"4 July 2020","externalUrl":null,"permalink":"/categories/centos/","section":"Categories","summary":"","title":"CentOS","type":"categories"},{"content":" The Yellowdog Updater, Modified, is the default utility to manage software packages on Red Hat Enterprise Linux. On Fedora (the upstream version of RHEL) Yum has been replaced with dnf but Red Hat decided to keep the Yum name for the RHEL releases. Although you\u0026rsquo;ll be using yum, under the hood you\u0026rsquo;re in fact using dnf which is why sometimes you\u0026rsquo;ll see references to dnf or dnf resources.\nThe Role of Repositories # Yum is designed to work with repositories which are depots of available software packages. Repositories makes it easy to keep your machine up to date, the maintainer of the repositories publishes updated packages and whenever you use yum to install software the most recent version is automatically used.\nAs an added benefit, yum manages package dependencies so you don\u0026rsquo;t have to deal with dependency hell. When a single package is installed, that same package will contain information about the required dependencies which yum will automatically install for you.\nOn Red Hat Enterprise Linux, you need to register the system on the Red Hat Customer Portal in order to obtain access to the Red Hat repositories. If you don\u0026rsquo;t register the system you will end up with no repositories at all, but you can create your own repo from the installation media (more on that later).\nRepositories are configured in the /etc/yum.repos.d/ directory as .repo files. Each file could contain multiple repositories, but each repository should have at least the following contents:\n[label] Identifies the specific repository. name= Specifies the name of the repository you want to use. baseurl= Contains the URL that points to the repo files. Can be HTTP, FTP, or a file path. Here\u0026rsquo;s an example of one of the default CentOS 8 repositories:\n[BaseOS] name=CentOS-$releasever - Base mirrorlist=http://mirrorlist.centos.org/?release=$releasever\u0026amp;arch=$basearch\u0026amp;repo=BaseOS\u0026amp;infra=$infra #baseurl=http://mirror.centos.org/$contentdir/$releasever/BaseOS/$basearch/os/ gpgcheck=1 enabled=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial Packages in Internet repositories are often signed with a GPG key which makes it possible to check whether they have been changed since the owner of the repository published them. If for some reason the repository security has been compromised, the GPG key signature will not match and the yum command will raise your attention about this.\nGPG-signed packages are not a requirement for internal or local repositories.\nCreating Your Own Repository # It\u0026rsquo;s fairly straightforward to setup your own repository in case you can\u0026rsquo;t or don\u0026rsquo;t want to register your RHEL system. You can put your own RPM packages (or those from the installation media) in a directory and publish that directory as a repository.\nIf you\u0026rsquo;re not using the installation media as the source for your own repository, you will need to run the createrepo command inside your repository directory to generate the metadata for the RPM files.\nYou can create repo from the installation media by mounting the ISO file persistently and creating the necessary .repo file.\nCreate the empty /repo directory and add the following line to the bottom of the /etc/fstab file to mount the ISO file at next boot:\n/path/to/file.iso /repo iso9660 defaults 0 0``` Next, mount the ISO file:\n[root@server1 ~]# mount /repo mount: /repo: WARNING: device write-protected, mounted read-only. [root@server1 ~]# Create the /etc/yum.repos.d/mycustom.repo file and add the following content:\n[AppStream] name=AppStream baseurl=file:///repo/AppStream gpgcheck=0 [BaseOS] name=BaseOS baseurl=file:///repo/BaseOS gpgcheck=0 Check the installed repo\u0026rsquo;s using yum repolist.\nWorking with Yum # To use repositories you need the yum command. Below you\u0026rsquo;ll find an overview of the most common yum tasks.\nCommand Description search Searches for the string you provide in package names and summaries. [what]provides */name Look for specific files inside a package. info Return more info about a package. install Install a package. remove Remove a package. list [all installed] group list List package groups. group install [–with-optional] Install all packages from a group. update Update packages or a specific packages. clean all Remove all stored metadata. history [undo ] List the command history / undo a specific command. To install a package you need the exact name, if you don\u0026rsquo;t know the exact name yum search can help you narrow that down. Remember it searches for the string you provide in package names and summaries, so you won\u0026rsquo;t have an exact match:\n[root@server1 ~]# yum search user ============ Name \u0026amp; Summary Matched: user ============= trousers-lib.x86_64 : TrouSerS libtspi library trousers-lib.i686 : TrouSerS libtspi library trousers-lib.x86_64 : TrouSerS libtspi library gnome-user-docs.noarch : GNOME User Documentation gnome-user-docs.noarch : GNOME User Documentation xdg-user-dirs.x86_64 : Handles user special directories xdg-user-dirs.x86_64 : Handles user special directories util-linux-user.x86_64 : libuser based util-linux utilities util-linux-user.x86_64 : libuser based util-linux utilities The yum provides command can help you find files inside a package which can be helpful if you know the name of the binary for example:\n[root@server1 ~]# yum provides */sepolicy policycoreutils-devel-2.9-3.el8.i686 : SELinux policy core policy devel utilities Repo : BaseOS Matched from: Filename : /usr/bin/sepolicy Filename : /usr/share/bash-completion/completions/sepolicy policycoreutils-devel-2.9-3.el8.x86_64 : SELinux policy core policy devel utilities Repo : BaseOS Matched from: Filename : /usr/bin/sepolicy Filename : /usr/share/bash-completion/completions/sepolicy We can obtain more information about a package using yum info:\n[root@server1 ~]# yum info nmap Available Packages Name : nmap Epoch : 2 Version : 7.70 Release : 5.el8 Architecture : x86_64 Size : 5.8 M Source : nmap-7.70-5.el8.src.rpm Repository : AppStream Summary : Network exploration tool and security scanner yum list | less will show us a list of available and installed packages.\nIf the repository name is shown, i.e. @AppStream, the package is available for installation in that repository. If @anaconda is shown, then that package has already been installed:\nInstalled Packages GConf2.x86_64 3.2.6-22.el8 @AppStream ModemManager.x86_64 1.10.4-1.el8 @anaconda Packages can be updated using yum update. The old version of a package is replaced with a new version, except for the kernel package. The newer kernel is installed along the old kernel so you can select the kernel you want to use in Grub when booting.\nTo make it easier to manage specific functionality instead of specific packages, we can work with package groups. Use yum groups list to show available package groups and yum groups info \u0026lt;groupname\u0026gt; to see what packages are in the specified group:\n[root@server1 ~]# yum groups list Last metadata expiration check: 0:00:12 ago on Wed 20 May 2020 17:01:09 +04. Available Environment Groups: Server Minimal Install Workstation Virtualization Host Custom Operating System Installed Environment Groups: Server with GUI Installed Groups: Container Management Headless Management Available Groups: .NET Core Development RPM Development Tools Development Tools Graphical Administration Tools Legacy UNIX Compatibility Network Servers Scientific Support Security Tools Smart Card Support System Tools [root@server1 ~]# yum groups info \u0026#34;System Tools\u0026#34; Last metadata expiration check: 0:00:25 ago on Wed 20 May 2020 17:01:09 +04. Group: System Tools Description: This group is a collection of various tools for the system, such as the client for connecting to SMB shares and tools to monitor network traffic. Default Packages: NetworkManager-libreswan chrony cifs-utils libreswan nmap openldap-clients samba-client setserial tigervnc tmux xdelta zsh Optional Packages: PackageKit-command-not-found aide amanda-client arpwatch You can use yum group install \u0026quot;System Tools\u0026quot; to install the Default Packages inside that group. If you need the Optional Packages as well, use the yum group install --with-optional \u0026quot;System Tools\u0026quot; command instead. Hidden groups can be revealed using yum groups list hidden, these are subgroups of specific groups.\nPackage Module Streams # To separate core operating system packages from user-space packages, we have two main repositories: BaseOS and AppStream.\nIn Red Hat Enterprise Linux 8 different versions of the same package can be offered using Package Module Streams which are found inside the AppStream repo.\nA module is a delivery mechanism for a set of RPM packages that belong together, and are typically organized around a specific version of an application, with all dependencies for that specific version.\nEach module can have one or more streams. A stream contains one specific version. Only one stream can be enabled at the same time which means that only one version can be installed on a system.\nEach module can have a default stream. Default streams make it easy to install packages using yum install without the need to learn about modules.\nModule streams can be active or inactive. Active streams allow the installation of the module version. Streams are active if marked as default or if they are enabled by a user, unless the whole module has been disabled or another stream of that module is enabled.\nModules can also have one or more profiles which are a list of packages installed together for a particular use case.\nLet\u0026rsquo;s have a look at some of the modules using yum module list:\n[root@server1 ~]# yum module list | grep -E \u0026#39;php|nginx\u0026#39; nginx 1.14 [d] common [d] nginx webserver nginx 1.16 common nginx webserver php 7.2 [d] common [d], devel, minimal PHP scripting language php 7.3 common, devel, minimal PHP scripting language Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled From the above output we see the nginx and php modules with their respective stream (1.14 \u0026amp; 1.16, 7.2 \u0026amp; 7.3) and their profiles (common, devel, minimal). You can see the same information for a specific module using yum module list \u0026lt;modulename\u0026gt;.\nFor each module we can get more detailed information using yum module info \u0026lt;modulename\u0026gt; or for a specific stream using yum module info \u0026lt;modulename:version\u0026gt;:\n[root@server1 ~]# yum module info php:7.3 Last metadata expiration check: 0:02:58 ago on Thu 21 May 2020 09:05:40 +04. Name : php Stream : 7.3 Version : 8010020191122191516 Context : 2430b045 Architecture : x86_64 Profiles : common, devel, minimal Repo : AppStream Summary : PHP scripting language Description : php 7.3 module ........... To investigate packages in a specific application stream, we use yum module info \u0026ndash;profile \u0026lt;modulename:version\u0026gt;. This will list the available profiles and the packages for that specific profile:\n[root@server1 ~]# yum module info --profile php:7.3 Last metadata expiration check: 0:06:09 ago on Thu 21 May 2020 09:05:40 +04. Name : php:7.3:8010020191122191516:2430b045:x86_64 common : php-cli : php-common : php-fpm : php-json : php-mbstring : php-xml devel : libzip : php-cli : php-common : php-devel : php-fpm : php-json : php-mbstring : php-pear : php-pecl-zip : php-process : php-xml minimal : php-cli : php-common Once you have the necessary information, we can enable a module stream and install the module. Every module has a default module stream, if that is the version you need then you don\u0026rsquo;t need to enable anything.\nIf, for example, we need php7.3 we would need to enable it before installing it. Note that this will also enable dependencies:\nYou can now install the module with with yum module install php.\nNotice in the above output how yum was complaining there was no default profile set for the PHP7.3 stream, so I needed to specify profile using yum module install php/minimal.\nNow that I have PHP7.3 installed, I can easily switch to PHP7.2. I don\u0026rsquo;t have to enable the PHP7.2 module stream since that one is the default.\n[root@server1 ~]# yum module reset php [root@server1 ~]# yum module install php:7.2/minimal [root@server1 ~]# yum distro-sync yum distro-sync ensures that all dependent packages which are not in the module itself are updated as well. The output of this command should be:\nDependencies resolved. Nothing to do. Complete! When using yum install packagename, the default module stream of a package will be installed if that module stream is enabled. You would only need to use yum module install packagename:version\u0026gt;/profile if you have specific version and/or profile requirements.\nQuerying Software Packages with RPM # There are two reasons why you should not use the rpm command to manage software packages.\nYum takes care of resolving package dependencies for you while rpm does not. There are two package databases on a RHEL system, the YUM database and the RPM database. When you install packages via yum, the YUM database is updated first and the information is synchronized with the RPM database. Installing packages with RPM will update the RPM database only. That doesn\u0026rsquo;t mean RPM isn\u0026rsquo;t useful. If you downloaded an RPM package you can still install it via yum install package.rpm.\nMore importantly, the rpm command enables us to get more information about packages:\nWe can use rpm -qa to show a list of all software that is installed on the system, similar to yum list installed. We can use grep on this command to find out specific package names: rpm -qa | grep php\nroot@server1 ~]# rpm -qa | grep php php-common-7.2.11-2.module_el8.1.0+209+03b9a8ff.x86_64 php-cli-7.2.11-2.module_el8.1.0+209+03b9a8ff.x86_64 Let\u0026rsquo;s find out more about the php-common package usin rpm -qi:\n[root@server1 ~]# rpm -qi php-common Name : php-common Version : 7.2.11 Release : 2.module_el8.1.0+209+03b9a8ff Architecture: x86_64 Install Date: Thu 21 May 2020 09:31:12 +04 Group : Unspecified Size : 6472361 License : PHP and BSD Signature : RSA/SHA256, Thu 05 Dec 2019 06:42:19 +04, Key ID 05b555b38483c65d Source RPM : php-7.2.11-2.module_el8.1.0+209+03b9a8ff.src.rpm Build Date : Thu 14 Nov 2019 08:15:12 +04 Build Host : x86-01.mbox.centos.org Relocations : (not relocatable) Packager : CentOS Buildsys \u0026amp;lt;bugs@centos.org\u0026gt; Vendor : CentOS URL : http://www.php.net/ Summary : Common files for PHP Description : The php-common package contains files used by both the php package and the php-cli package. We can list the files inside the package using rpm -ql:\n[root@server1 ~]# rpm -ql php-common /etc/php.d /etc/php.d/20-bz2.ini /etc/php.d/20-calendar.ini /etc/php.d/20-ctype.ini ..... Or, we can list only the documentation using rpm -qd, or the configuration files using rpm -qc:\n[root@server1 ~]# rpm -qd php-common /usr/share/doc/php-common/CODING_STANDARDS /usr/share/doc/php-common/CREDITS ..... [root@server1 ~]# rpm -qc php-common /etc/php.d/20-bz2.ini /etc/php.d/20-calendar.ini /etc/php.d/20-ctype.ini ... If you have a filename and want to know what package it belongs to, use rpm -qf:\n[root@server1 ~]# rpm -qf /bin/bash bash-4.4.19-10.el8.x86_64 [root@server1 ~]# rpm -qf /bin/lsblk util-linux-2.32.1-17.el8.x86_64 All the above queries were used on the RPM database and what we were querying were installed packages. Sometimes it makes sense to query an RPM package file before installing it, in that case we need to add the -p option in addition to any of the previous mentioned options. We can use yumdownloader to download a specific package from our repository to run an RPM query against it before installing it:\n[root@server1 ~]# yum whatprovides */yumdownloader Last metadata expiration check: 0:00:38 ago on Thu 21 May 2020 20:34:10 +04. yum-utils-4.0.8-3.el8.noarch : Yum-utils CLI compatibility layer Repo : BaseOS Matched from: Filename : /usr/bin/yumdownloader [root@server1 ~]# yum install yum-utils -y .... [root@server1 ~]# yumdownloader httpd [root@server1 ~]# rpm -qpi httpd-2.4.37-16.module_el8.1.0+256+ae790463.x86_64.rpm Name : httpd Version : 2.4.37 Release : 16.module_el8.1.0+256+ae790463 .... We can also query packages directly from the repository instead of downloading the package first. Use the repoquery command for this.\nCommand Description rpm -qf Use a filename to find the specific RPM package the file belongs to rpm -ql Provide a list of files inside the RPM package rpm -qi Provide package information rpm -qd Show all documentation available in the package rpm -qc Show all configuration files rpm -q -scripts Show scripts that are used in the package rpm -qp Query individual .rpm files instead of the RPM database rpm -qR Show package dependencies rpm -V Shows which parts of a package has been changed since installation. rpm -Va Verifies all installed packages and shows which part of the package has been changed since installation. rpm -qa List all installed packages ","date":"4 July 2020","externalUrl":null,"permalink":"/package-management-with-yum-understanding-repos-groups-package-module-streams-and-rpm-queries/","section":"Blog","summary":"","title":"Package Management with Yum: Understanding Repo’s, Groups, Package Module Streams and RPM Queries","type":"posts"},{"content":"","date":"4 July 2020","externalUrl":null,"permalink":"/tags/package-module-stream/","section":"Tags","summary":"","title":"Package Module Stream","type":"tags"},{"content":"","date":"4 July 2020","externalUrl":null,"permalink":"/tags/repo/","section":"Tags","summary":"","title":"Repo","type":"tags"},{"content":"","date":"4 July 2020","externalUrl":null,"permalink":"/tags/yum/","section":"Tags","summary":"","title":"Yum","type":"tags"},{"content":"","date":"28 June 2020","externalUrl":null,"permalink":"/tags/centos8/","section":"Tags","summary":"","title":"Centos8","type":"tags"},{"content":" In the previous post we learned how the check the runtime configuration of the network using the ip command. To make persistent changes to the network configuration that will survive a reboot, we need to use either nmcli or nmtui.\nNetworking is managed by the NetworkManager service. When the NetworkManager service comes up, it reads the network card configuration scripts that are located in /etc/sysconfig/network-scripts and that have a name which starts with ifcgf followed by the name of the network card.\nYou can check the status of the NetworkManager using systemctl status NetworkManager.\nOn RHEL 8, we differentiate between a device and a connection as follows:\nA device is a network interface card A connection is the configuration that is applied to the device You can create multiple connections for one device and manage the connections we assign to devices using nmcli or nmtui.\nRequired Permissions # The root user can make modifications to the network configuration and so can regular users if they are logged in to the local console: if a regular user is using the system keyboard to enter either a graphical or text-based console, some permissions to change the network configuration are granted. This is because users are supposed to be able to connect their local system to a network.\nUsers that have used ssh to connect to a server are not allowed to change the network configuration.\nYou can check the current permissions using the nmcli gen permissions command:\nConfiguring the Network with nmcli # We can use the nmcli command to make persistent changes to our network configuration. nmcli will write your configuration to the network card configuration scripts located in /etc/sysconfig/network-scripts from where it will be read by the NetworkManager service during boot time or when restarting the service.\nActive and inactive connections can be shown using the nmcli con show command. Inactive connections are not assigned to a device:\nOnce you have an overview of the connections, you can see the details of a connection using nmcli con show _connectionname_, e.g. nmcli con show ens1. This will show all properties of the given connection. Check man nm-settings to find out what these settings do exactly.\nJust as with connections, we can see the currently configured devices and their status: nmcli dev status\nWe use nmcli dev show devicename to reveal the settings for a specific device: Creating Network Connections # We can use the nmcli con add command to create a new connection on a specific device. Bash completion and the nmcli-examples man page can help you on this. I\u0026rsquo;ll list two examples below, one dhcp configured connection and one static configuration.\n[student@server1 ~]$ nmcli con add con-name dhcp-config type ethernet ifname ens1 ipv4.method auto [student@server1 ~]$ nmcli con add con-name static-config type ethernet ifname ens1 autoconnect no ip4 10.0.0.10/24 gw4 10.0.0.1 ipv4.method manual Modifying Connection Parameters # nmcli con mod allows us to modify the connections we added earlier.\nLet\u0026rsquo;s make sure our static connection automatically connects:\n[student@server1 ~]$ nmcli con mod static-config autoconnect yes We\u0026rsquo;ll also add a DNS server, notice how i\u0026rsquo;m using ipv4 and not ip4 like when adding a connection:\n[student@server1 ~]$ nmcli con mod static-config ipv4.dns 10.0.0.10 We can add a secondary DNS server using the + sign:\n[student@server1 ~]$ nmcli con mod static-config +ipv4.dns 8.8.8.8 Let\u0026rsquo;s also change the current IP address and add a second IP address:\n[student@server1 ~]$ nmcli con mod static-config ipv4.addresses 10.0.0.100/24 [student@server1 ~]$ nmcli con mod static-config +ipv4.addresses 10.20.30.40/16 When we are done with our modifications, we should activate our changes:\n[student@server1 ~]$ nmcli con up static-config Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5) [student@server1 ~]$ Configuring the Network with nmtui # nmtui is a textual user interface for the fairly complicated syntax of the nmcli command. Everything that can be done with nmcli can also be done with nmtui so I won\u0026rsquo;t cover it too much as it\u0026rsquo;s pretty much self-explanatory.\nWorking with Network Configuration Files # If you don\u0026rsquo;t like making configuration changes using nmcli or nmtui you can directly edit the network interface card configuration file itself. After making changes to the configuration file, use the nmcli con up command to activate the new configuration.\nYou can set both a fixed IP address and a dynamic IP address in one network configuration. Set the BOOTPROTO option to in the configuration file to dhcp while also specifying an IP address and network prefix. You can also do this from nmtui by making sure the IPv4 configuration is set to Automatic while also specifying an IP address.\nHostname and Name Resolution # Hostnames are used to communicate with other hosts. A hostname consists of the name of the host and the DNS domain in which they reside, these two parts together make up for the fully qualified domain name (FQDN), e.g. server1.example.com. An FQDN would provide a unique identity on the internet.\nHostnames # We can use different ways to set the hostname:\nUse nmtui and select the Change Hostname option Use hostnamectl set-hostname Edit the /etc/hostname configuration file After setting the hostname, you can use hostnamectl status to show the current hostname:\n[root@server1 ~]# hostnamectl set-hostname server1.example.local [root@server1 ~]# hostnamectl status Static hostname: server1.example.local We can configure hostname resolution in the /etc/hosts file.\nThe first column has the IP address of the host, the second specifies the hostname (either short or FQDN). If it has more than one name (short and FQDN) the second column must be the FQDN and the third one the alias or shortname:\nbash[root@server1 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 10.0.0.2 server2.example.local server2 Definitions in the /etc/hosts file will be applied before the hostname in DNS resolution is used. This priority is set in /etc/nsswitch.conf where files is a reference to the /etc/hosts file.\nhosts: files dns myhostname DNS Name Resolution # In order to communicate with other hosts on the Internet we will need DNS.\nWe already know how to set a DNS server using nmcli or nmtui.\nThe NetworkManager reads the configuration script in /etc/sysconfig/network-scripts and pushes the DNS configuration to /etc/resolv.conf.\nIf you edit the /etc/resolv.conf file manually, your changes will be overwritten by NetworkManager the next time it starts.\nIt\u0026rsquo;s recommended to setup at least two DNS Name Servers for redundancy:\nUse nmtui Set the DNS1 and DNS2 parameters in the ifcfg network configuration file Use DHCP Use nmcli con mod \u0026lt;connection name\u0026gt; [+]ipv4.dns \u0026lt;ip-of-dns\u0026gt; If the connection is configured to get the configuration from a DHCP server then the DNS server is also set via DHCP. If you don\u0026rsquo;t want that to happen you have two options:\nEdit the ifcfg configuration file to include the option PEERDNS=no Use nmcli cod mod \u0026lt;connection name\u0026gt; ipv4.ignore-auto-dns yes To verify hostname resolution use the getent hosts \u0026lt;hostname\u0026gt; command. This searches in both /etc/hosts and DNS to resolve the specified hostname.\n","date":"28 June 2020","externalUrl":null,"permalink":"/configuring-the-network-on-rhel-8-with-nmcli-and-nmtui/","section":"Blog","summary":"","title":"Configuring the Network on RHEL 8 with nmcli and nmtui","type":"posts"},{"content":"","date":"28 June 2020","externalUrl":null,"permalink":"/tags/network-configuration/","section":"Tags","summary":"","title":"Network Configuration","type":"tags"},{"content":"","date":"28 June 2020","externalUrl":null,"permalink":"/tags/nmcli/","section":"Tags","summary":"","title":"Nmcli","type":"tags"},{"content":"","date":"28 June 2020","externalUrl":null,"permalink":"/tags/nmtui/","section":"Tags","summary":"","title":"Nmtui","type":"tags"},{"content":"","date":"25 June 2020","externalUrl":null,"permalink":"/tags/network-validation/","section":"Tags","summary":"","title":"Network Validation","type":"tags"},{"content":"","date":"25 June 2020","externalUrl":null,"permalink":"/tags/rhel8/","section":"Tags","summary":"","title":"Rhel8","type":"tags"},{"content":" Networking is one of the essential items on a server. On Red Hat Enterprise Linux 8, networking is managed by the NetworkManager service. We\u0026rsquo;ll cover new tools that were introduced to help manage networks during runtime and how to make make the configuration persistent.\nValidating Network Addresses and Interfaces # In RHEL 8, the default names for network cards are based on firmware, device topology, and device firmware. Network card names will always consist of the following parts:\nEthernet interfaces begin with en, WLAN interfaces begin with wl and WWAN interfaces begin with ww The next part of the name represents the type of adapter.\nAn o is used for onboard, s for hotplug slot, p for PCI location. A number is used to represent an index, ID or port. e.g. enp3p1 would indicate an ethernet device on PCI slot 3, port 1.\nApart from this default device naming scheme, BIOS device naming can be used as well if the biosdevname package is installed.\nValidating Network Address Configuration # To verify the runtime configuration of the network, we can use the ip utility. We can use this utility to monitor many aspects of networking:\nip addr to show and configure network addresses ip route to show and configure routing information ip link to show and configure network link state We can use the ip addr show command to show the current network settings:\n[student@server1 ~]$ ip addr show 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens1: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:d4:91:88 brd ff:ff:ff:ff:ff:ff inet 192.168.122.237/24 brd 192.168.122.255 scope global dynamic noprefixroute ens1 valid_lft 3504sec preferred_lft 3504sec inet6 fe80::ccd3:f5b4:4faf:2321/64 scope link noprefixroute valid_lft forever preferred_lft forever [student@server1 ~]$ This lists all network interfaces on the system.\nIn the above case we see 2 network interfaces, the loopback interface lo and the hot-pluggable device ens1. The loopback interface is used for IP communication between processes on the machine.\nWe can see:\nThe current link state:\n2: ens1: mtu 1500 qdisc fq_codel state UP group default qlen 1000 The MAC address configuration:\nlink/ether 52:54:00:d4:91:88 the IPv4 configuration:\ninet 192.168.122.237/24 and the IPv6 configuration:\ninet6 fe80::ccd3:f5b4:4faf:2321/64 Every interface automatically gets an IPv6 address which can only be used for communication on the local network. Such addresses start with fe80.\nIf you are interested in the link status only of the network interfaces, you can use the ip link show command. You can reveal statistics on received packets (RX) and transmitted packets (TX) by adding the -s option: ip -s link show\nValidating Routing # Routing is required for every network that needs to communicate to devices on other networks. For that to work, the network but have at least one default router or gateway. The default gateway must always be on the same network or subnet. You can check which gateway is being used with the ip route show command:\n[student@server1 ~]$ ip route show default via 192.168.122.1 dev ens1 proto dhcp metric 100 192.168.122.0/24 dev ens1 proto kernel scope link src 192.168.122.237 metric 100 The most important part is the first line that shows the default route goes through 192.168.122.1 and also shows that device ens1 must be used to access that gateway. We can also see that this route was assigned by dhcp.\nIn case of multiple routes, the route with the lowest metric will be used.\nValidating Port and Service Availability # Network issues can be related to the local IP address or router settings but can also come from network ports that are not available on the server.\nWe use the ss command to verify the availability of ports. By using ss -lt we can see the listing TCP ports on the local system and by using ss -lu we see the UDP ports. We can combine the commands to ss -tul:\nNotice that some ports/services are only listening on the loopback address while others are listening on 0.0.0.0 (all IPv4 addresses) or on [::] (all IPv6 addresses).\n","date":"25 June 2020","externalUrl":null,"permalink":"/validating-network-configuration-on-red-hat-enterprise-linux-8/","section":"Blog","summary":"","title":"Validating Network Configuration on Red Hat Enterprise Linux 8","type":"posts"},{"content":" If you\u0026rsquo;re comfortable using the command line, you\u0026rsquo;re probably aware there are different ways you can push your Kinsta Staging site to Live without overwriting the Live database. This can be useful when you need to push code changes, and at the same time need to leave the Live database in its current state.\nThis post will cover how you can achieve this using GitLab CI/CD and assumes you\u0026rsquo;re familiar with version control using git, and using the Kinsta Staging environment for testing and development. If you\u0026rsquo;re developing locally, you should be able to apply the information in this post as well.\nThe end result will be that whenever you merge changes into the master branch on Staging and push those changes to the remote master branch, GitLab will automatically deploy the updated master branch to Kinsta Live.\nAs an added bonus you will also have implemented version control so you can easily roll back any changes you make to your code.\nLet\u0026rsquo;s get started!\nCreate a Kinsta Site and GitLab Project # If you don\u0026rsquo;t have a Kinsta site already, create one following the instructions here. In this case, I\u0026rsquo;ll create an empty site with the name gitlabautodeploy and add WordPress later on.\nNext, we\u0026rsquo;ll create an empty project on Gitlab. This repository will contain our website\u0026rsquo;s files. In my case I\u0026rsquo;ll create a private project.\nCloning, Configuring and Populating the GitLab Repo on Kinsta # Setting up SSH Keys # In order to clone the repository on our Live environment, we\u0026rsquo;ll have to add our SSH public key to our GitLab SSH Keys.\nConnect to Live via SSH and copy the output of the below command, then add it your GitLab account settings following the instructions here.\ngitlabautodeploy@gsY-gitlabautodeploy:~$ cat .ssh/id_rsa.pub This will allow us to access and clone the repository over SSH without the need for authentication.\nAdd the same public key to your SSH Keys in My Kinsta This step is equally important as the next. Without this step GitLab will not be able to connect to the Live environment.\nNext, we want to add our SSH private key as a variable to the repository\u0026rsquo;s CI/CD Pipeline settings. Copy the output of the below command:\ngitlabautodeploy@gsY-gitlabautodeploy:~$ cat .ssh/id_rsa Under Settings \u0026gt; CI/CD in your GitLab repo, add a new Variable containing your private key. Give the variable the name SSH_PRIVATE_KEY, paste the private key in the Value field, and add the variable.\nThe private key will allow GitLab to connect to our Kinsta Live environment and deploy the changes we\u0026rsquo;ve pushed from Staging.\nClone and Configure the Repo # We can now clone the empty repository onto Kinsta Live. Go to your repository overview on GitLab and click the blue Clone dropdown, select Clone with SSH and copy the value.\nPaste the value after the git clone command inside the home directory on Live and execute it:\ngitlabautodeploy@gsY-gitlabautodeploy:~$ git clone git@gitlab.com:joerismissaert/gitlabautodeploy.git Once the cloning has finished, you\u0026rsquo;ll notice an extra folder was created with the name of our repository:\ngitlabautodeploy@gsY-gitlabautodeploy:~$ ls -lh total 19K drwxr-xr-x 3 gitlabautodeploy www-data 3 May 14 08:57 gitlabautodeploy By default, Kinsta serves websites from the ~/public folder. You can reach out to support to have this changed, but in this case I\u0026rsquo;ll delete the existing public folder and rename my cloned repository to public:\ngitlabautodeploy@gsY-gitlabautodeploy:~$ rm -rf public/ gitlabautodeploy@gsY-gitlabautodeploy:~$ mv gitlabautodeploy public gitlabautodeploy@gsY-gitlabautodeploy:~$ In the next step we\u0026rsquo;ll configure our repo:\ngitlabautodeploy@gsY-gitlabautodeploy:~$ cd public gitlabautodeploy@gsY-gitlabautodeploy:~/public$ git config --global user.email \u0026#34;youremail@address.com\u0026#34; gitlabautodeploy@gsY-gitlabautodeploy:~/public$ git config --global user.name \u0026#34;Joeri\u0026#34; gitlabautodeploy@gsY-gitlabautodeploy:~/public$ Install and Configure WordPress # We are now ready to install and configure WordPress. Go ahead and download the latest version of WordPress inside your repository on Live:\ngitlabautodeploy@gsY-gitlabautodeploy:~/public$ wp core download Downloading WordPress 5.4.1 (en_US)… md5 hash verified: 346afd52e893b2492e5899e4f8c91c43 Success: WordPress downloaded. gitlabautodeploy@gsY-gitlabautodeploy:~/public$ If you know how to setup WordPress manually, great! Else you can use the Web Interface by going to the Primary Domain for your Kinsta site. Follow the instructions and configure the site.\nConfiguring the GitLab CI/CD Pipeline # We need to add a YAML file called .gitlab-ci.yml to our repository that contains instructions for GitLab to deploy our changes to Live. When GitLab detects this file, it will start up a Docker container and configure the Docker environment based on the instructions in our file to be able to connect to our Live environment and deploy the updated master branch.\nCreate the .gitlab-ci.yml file and add the following:\nbefore_script: - apt-get update -qq - apt-get install -qq git # Setup SSH deploy keys - \u0026#39;which ssh-agent || ( apt-get install -qq openssh-client )\u0026#39; - eval $(ssh-agent -s) - ssh-add \u0026lt;(echo \u0026#34;$SSH_PRIVATE_KEY\u0026#34;) - mkdir -p ~/.ssh - \u0026#39;[[ -f /.dockerenv ]] \u0026amp;\u0026amp; echo -e \u0026#34;Host *\\n\\tStrictHostKeyChecking no\\n\\n\u0026#34; \u0026amp;gt; ~/.ssh/config\u0026#39; deploy_live: type: deploy environment: name: Live url: gitlabautodeploy.kinsta.cloud script: - ssh kinstauser@IPADDRESS -p PORTNUMBER \u0026#34;cd /www/gitlabautodeploy_941/public \u0026amp;\u0026amp; git checkout master \u0026amp;\u0026amp; git pull origin master \u0026amp;\u0026amp; exit\u0026#34; only: - master Pay attention to the deploy_live block where you will want to modify the url, ssh connection string and the path in the command. Nothing is actually being done with the url value in this case though.\nThe only: - master part refers to the branch of your repository that should trigger this script when changes have been pushed to it. If you change this, make sure to change the SSH command accordingly.\nYou could add a deploy_staging block that contains the SSH connection string for the Staging environment and the only: - staging block list item so that a git push to the equally named staging branch would trigger a deploy to Kinsta\u0026rsquo;s Staging environment from your local development environment for example.\nInitial Commit # Once the site is set up and you\u0026rsquo;ve confirmed it\u0026rsquo;s working properly, we can commit and push our changes to the remote repository on GitLab.\nWe\u0026rsquo;ll create a new branch for our repository called v0.1, then start tracking all the files we added to our Repo, commit our changes, and push our changes to the remote origin on the new branch.\ngitlabautodeploy@gsY-gitlabautodeploy:~/public$ git checkout -b v0.1 Switched to a new branch \u0026#39;v0.1\u0026#39; gitlabautodeploy@gsY-gitlabautodeploy:~/public$ git add . gitlabautodeploy@gsY-gitlabautodeploy:~/public$ git commit -a -m \u0026#39;Initial Commit\u0026#39; gitlabautodeploy@gsY-gitlabautodeploy:~/public$ git push origin v0.1 Creating Staging and Testing the Deployment # We\u0026rsquo;re all set up, let\u0026rsquo;s create the Staging environment and run some tests.\nFollow the instructions here to create the Staging environment and connect to Staging using SSH.\nWe should still be on the new branch we created earlier and we\u0026rsquo;ll see the wp-config.php file has been modified. This was done by Kinsta\u0026rsquo;s Staging creation script and is completely normal.\ngitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ git status On branch v0.1 Changes not staged for commit: (use \u0026#34;git add …\u0026#34; to update what will be committed) (use \u0026#34;git restore …\u0026#34; to discard changes in working directory) modified: wp-config.php no changes added to commit (use \u0026#34;git add\u0026#34; and/or \u0026#34;git commit -a\u0026#34;) gitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ Before we go and commit the change, let\u0026rsquo;s add a simple plugin: classic-editor\ngitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ wp plugin install classic-editor Downloading installation package from https://downloads.wordpress.org/plugin/classic-editor.1.5.zip… Unpacking the package… Installing the plugin… Plugin installed successfully. Success: Installed 1 of 1 plugins. gitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ Add all untracked files, commit the changes and push the changes to the remote origin on the v0.1 branch:\ngitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ git add . gitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ git commit -a -m \u0026#39;Installed classic-editor plugin\u0026#39; gitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ git push origin v0.1 gitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ Now let\u0026rsquo;s switch to our master branch, merge the v0.1 branch and push the merge to the remote master branch:\ngitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ git checkout master Switched to branch \u0026#39;master\u0026#39; gitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ git merge v0.1 gitlabautodeploy@XJl-staging-gitlabautodeploy:~/public$ git push origin master Go to your GitLab project: CI / CD \u0026gt; Jobs, where you\u0026rsquo;ll see the deployment running:\nYou can click on the Status to see the deployment in action: If everything went well, the Status will change from running to passed and your live site now has the classic-editor plugin installed.\n","date":"13 June 2020","externalUrl":null,"permalink":"/auto-deploy-branch-changes-to-kinsta-using-gitlab-ci-cd/","section":"Blog","summary":"","title":"Auto Deploy Branch Changes to Kinsta using GitLab CI/CD","type":"posts"},{"content":"","date":"13 June 2020","externalUrl":null,"permalink":"/tags/ci/cd/","section":"Tags","summary":"","title":"CI/CD","type":"tags"},{"content":"","date":"13 June 2020","externalUrl":null,"permalink":"/tags/git/","section":"Tags","summary":"","title":"Git","type":"tags"},{"content":"","date":"13 June 2020","externalUrl":null,"permalink":"/tags/gitlab/","section":"Tags","summary":"","title":"Gitlab","type":"tags"},{"content":"","date":"13 June 2020","externalUrl":null,"permalink":"/tags/kinsta/","section":"Tags","summary":"","title":"Kinsta","type":"tags"},{"content":"","date":"13 June 2020","externalUrl":null,"permalink":"/categories/lab/","section":"Categories","summary":"","title":"Lab","type":"categories"},{"content":"","date":"2 June 2020","externalUrl":null,"permalink":"/tags/file-permissions/","section":"Tags","summary":"","title":"File Permissions","type":"tags"},{"content":"","date":"2 June 2020","externalUrl":null,"permalink":"/tags/lab/","section":"Tags","summary":"","title":"Lab","type":"tags"},{"content":" In the previous two posts we learned about basic and advanced permissions, Access Control Lists, umask and extended-attributes.\nLet\u0026rsquo;s put this knowledge to work by setting up a shared group environment\nLab Objectives # Create 4 random users and two groups: finance \u0026amp; sales.\nAdd two users to the first group and two to the second group. Create 2 directories: /data/finance \u0026amp; /data/sales.\nMake the group sales the group owner of the directory sales, and make the group finance the group owner of the directory finance. Make sure the user owner of the directories is root.\nGroup owners should have full access to their directories and, no permissions should be assigned to the others entity. The others entity should have no permissions on newly created files and directory within the entire /data structure. Set the permissions so that members of the group sales can read files in the /data/finance directory, and members of the group finance can read files in the /data/sales directory. Ensure that all new files and directories inherit the group owner of their respective directory. Ensure that users are only allowed to remove files of which they are the owner. Lab Solution # Objective 1 # We\u0026rsquo;ll create the finance and sales group, as well as the users betty, bob, bill and bea. We\u0026rsquo;ll add betty and bob to finance, and bill and bea to sales.\n[root@server1 ~]# groupadd finance \u0026amp;\u0026amp; groupadd sales [root@server1 ~]# for i in betty bob bill bea; do useradd $i; done [root@server1 ~]# usermod -aG finance betty \u0026amp;\u0026amp; usermod -aG finance bob [root@server1 ~]# usermod -aG sales bill \u0026amp;\u0026amp; usermod -aG sales bea [root@server1 ~]# Objective 2 # [root@server1 ~]# groupadd finance \u0026amp;\u0026amp; groupadd sales [root@server1 ~]# for i in betty bob bill bea; do useradd $i; done [root@server1 ~]# usermod -aG finance betty \u0026amp;\u0026amp; usermod -aG finance bob [root@server1 ~]# usermod -aG sales bill \u0026amp;\u0026amp; usermod -aG sales bea [root@server1 ~]# mkdir -p /data/finance /data/sales [root@server1 ~]# chown :finance /data/finance [root@server1 ~]# chown :sales /data/sales [root@server1 ~]# chown root /data/finance [root@server1 ~]# chown root /data/sales [root@server1 ~]# chmod g+rwx /data/finance [root@server1 ~]# chmod g+rwx /data/sales [root@server1 ~]# chmod o-rwx /data/finance [root@server1 ~]# chmod o-rwx /data/sales [root@server1 ~]# ls -l /data total 0 drwxrwx---. 2 root finance 6 May 10 20:16 finance drwxrwx---. 2 root sales 6 May 10 20:16 sales [root@server1 ~]# Objective 3 # We set the default ACL for the others entity to no permissions recursively, these permissions will apply to all newly created files and directories. Then apply a regular ACL.\n[root@server1 ~]# setfacl -R -m d:o::- /data [root@server1 ~]# setfacl -R -m o::- /data We\u0026rsquo;ll make sure our two groups can access the /data directory and any new sub-directory.\n[root@server1 /]# setfacl -m d:g:sales:rx,d:g:finance:rx /data [root@server1 /]# setfacl -m g:sales:rx,g:finance:rx /data You can verify your work using getfacl.\nObjective 4 # We start by setting the default ACLs:\n[root@server1 ~]# setfacl -m d:g:sales:rx,d:g:finance:rwx /data/finance/ [root@server1 ~]# setfacl -m d:g:finance:rx,d:g:sales:rwx /data/sales And afterward, we set the ACLs for the current files and directories (although there are none, it\u0026rsquo;s best practice):\n[root@server1 ~]# setfacl -R -m g:sales:rx,g:finance:rwx /data/finance [root@server1 ~]# setfacl -R -m g:finance:rx,g:sales:rwx /data/sales Remember that we need to set execute permissions on the directory level in order to be able to read a file contained inside that directory.\nObjective 5 # We set the SGUID permission on both the sales and finance directories:\n[root@server1 ~]# chmod g+s /data/sales [root@server1 ~]# chmod g+s /data/finance [root@server1 ~]# ls -l /data total 0 drwxrws---+ 2 root finance 6 May 10 20:16 finance drwxrws---+ 2 root sales 6 May 10 20:16 sales Objective 6 # We set the Sticky Bit permission on both directories:\n[root@server1 ~]# chmod +t /data/sales [root@server1 ~]# chmod +t /data/finance [root@server1 ~]# ls -l /data total 0 drwxrws--T+ 2 root finance 6 May 10 20:16 finance drwxrws--T+ 2 root sales 6 May 10 20:16 sales Verifying the solution # We\u0026rsquo;ll test the applied permissions for some of the users we created.\nLogin as bill from sales and create a file:\n[root@server1 ~]# su - bill [bill@server1 ~]$ echo \u0026#39;Hello!\u0026#39; \u0026amp;gt; /data/sales/bill_file [bill@server1 ~]$ exit [root@server 1 ~]# Let\u0026rsquo;s check what betty from finance can do:\n[root@server1 ~]# su - betty [betty@server1 ~]$ cd /data [betty@server1 data]$ ls finance sales [betty@server1 data]$ cd finance/ [betty@server1 finance]$ ls [betty@server1 finance]$ touch betty_file [betty@server1 finance]$ ls -l total 0 -rw-rw----+ 1 betty finance 0 May 10 20:45 betty_file [betty@server1 finance]$ cd /data/sales/ [betty@server1 sales]$ ls bill_file [betty@server1 sales]$ cat bill_file Hello! [betty@server1 sales]$ rm bill_file rm: remove write-protected regular file \u0026#39;bill_file\u0026#39;? y rm: cannot remove \u0026#39;bill_file\u0026#39;: Permission denied [betty@server1 sales]$ touch betty_sales touch: cannot touch \u0026#39;betty_sales\u0026#39;: Permission denied [betty@server1 sales]$ We see that betty can create new files in /data/finance, and the group owner is set to finance. Betty can read files in /data/sales but can\u0026rsquo;t delete or create new files.\nLet\u0026rsquo;s login as bob from finance:\n[root@server1 /]# su - bob [bob@server1 ~]$ cd /data/finance/ [bob@server1 finance]$ cat betty_file Hello! [bob@server1 finance]$ rm betty_file rm: cannot remove \u0026#39;betty_file\u0026#39;: Operation not permitted [bob@server1 finance]$ Bob can do everything Betty can, but won\u0026rsquo;t be able to delete files that don\u0026rsquo;t belong to him.\nImportant! # The Sticky Bit permission can not be inherited from the parent directory. This means that if Bob creates a new directory in /data/finance but doesn\u0026rsquo;t change the permissions on that directory, then Betty will be able to read, write and execute files in Bob\u0026rsquo;s new directory.\nBob can set any permission he likes on his new directory since he\u0026rsquo;s the user owner.\n","date":"2 June 2020","externalUrl":null,"permalink":"/lab-permissions-in-practice/","section":"Blog","summary":"","title":"Lab: Permissions In Practice","type":"posts"},{"content":"","date":"2 June 2020","externalUrl":null,"permalink":"/categories/linux/","section":"Categories","summary":"","title":"Linux","type":"categories"},{"content":"","date":"1 June 2020","externalUrl":null,"permalink":"/tags/access-control-lists/","section":"Tags","summary":"","title":"Access Control Lists","type":"tags"},{"content":" Access Control Lists allow you to add permissions to more than one user or group on the same file or directory, and allows you to set default permissions for newly created files and directories.\nManaging ACLs # It\u0026rsquo;s possible you need need to add file system support for ACLs during boot time by adding the acl mount option the the /etc/fstab file. That would be the case when you\u0026rsquo;re seeing the Operation not supported message when applying an ACL.\nViewing and Changing ACL Settings # The ls -l command doesn\u0026rsquo;t show any existing ACLs, it will show a + after the permission listing to indicate ACL permissions have been set:\ndrwxrwsr-x+ 2 joeri joeri 4096 May 9 16:14 somedirectory/ To show the current ACL settings, use the getfacl command:\n$ getfacl somedirectory # file: somedirectory # owner: joeri # group: joeri # flags: -s- user::rwx group::rwx group:joeri:rwx mask::rwx other::r-x You can see the permissions are shown for the usual three entities.\nWe use the setfacl command to add an ACL:\nsetfacl -m g:users:rwx somedirectory - Sets the RWX permissions for the group users setfacl -m u:joeri:rwx somedirectory - Sets the RWX permissions for the user joeri setfacl -m g:sales:- somedirectory - Removes all permissions for the group sales setfacl -R -m o::- somedirectory - Recursively removes all permissions for the others entitiy setfacl -x g:sales somedirectory - Removes the ACL for the group sales. Working with default ACLs # Default ACLs allows you to enable inheritance. The default ACL will set the permissions on all new items that are created in a given directory, but will not change the permissions for existing files and subdirectories.\nUsually you will set ACLs twice:\nsetfacl -R -m to modify the ACL for current files.\nsetfacl -m d: to take care of all new items.\nsetfacl -m d:g:sales:rx somedirectory Would set the read and execute permissions for the sales group on all new files and subdirectories.\nAlways set basic permissions first before applying ACLs, and avoid changing the basic permissions after applying ACLs.\nSetting Default Permissions with umask # When creating new files default permissions are set by the shell determined by the umask shell value which is applied during login.\nThe umask numerical value is substracted from the maximum permissions a file or directory can get, 666 and 777 respectively.\ne.g. a umask setting of 022 will result in 644 for files (6-0, 6-2, 6-2) and 755 for directories (7-0, 7-2, 7-2).\nThere are two ways to change the umask setting, either for all users or for individual users. The value is set in /etc/login.defs for all users and in ~.bashrc_profile for an individual user.\nValue Files Directories 0 RW ALL 1 RW RW 2 R RX 3 R R 4 W WX 5 W WX 6 - X 7 - - Working with User-Extended Attributes # When working with permissions, there\u0026rsquo;s always been a relationship between a user or a group, and a file or directory. This isn\u0026rsquo;t the case with User-Extended Attributes, they do their work regardless of the user or group who accesses a file.\nJust like ACLs, it\u0026rsquo;s possible you need to add the user_xattr mount option.\nThere\u0026rsquo;s a lot of attributes you can apply to a file, but I\u0026rsquo;ll only cover one in particular which seems the most useful to me: The immutable attribute.\nThe immutable attribute makes the file immutable, no changes can be made at all to file.\nYou can set an attribute using the chattr command:\nchattr +i somefile adds the immutable attribute to the file.\nSimilarly, you can remove it using chattr -i somefile.\nTo list the current applied attributes use the lsattr command on a file.\n","date":"1 June 2020","externalUrl":null,"permalink":"/access-control-lists-umask-user-extended-attributes-101/","section":"Blog","summary":"","title":"Access Control Lists, umask \u0026 User-Extended Attributes 101","type":"posts"},{"content":"","date":"1 June 2020","externalUrl":null,"permalink":"/tags/acl/","section":"Tags","summary":"","title":"ACL","type":"tags"},{"content":"","date":"1 June 2020","externalUrl":null,"permalink":"/tags/user-extended-attributes/","section":"Tags","summary":"","title":"User-Extended Attributes","type":"tags"},{"content":" To get access to files on Linux, permissions are used. These permissions are assigned to three entities: the file owner, the group owner, and the others entity.\nManaging File Ownership # Displaying Ownership # Every file and directory on Linux has two owners: a user owner and a group owner. There is also the \u0026ldquo;others\u0026rdquo; entity. The user , group and others are shown when listing permissions with the ls -l command. The below test file has read/write permissions for the user owner and group owner and read permissions for anyone else.\n$ ls -l # -rw-rw-r--. 1 joeri joeri 0 May 9 14:29 test File ownership is checked in a specific order. The shell checks:\nif you are the user owner. If you are, the shells stops checking here. if you have obtained permissions through user-assigned Access Control Lists (more on that in a next post). if you are a member of the group owner. if you have obtained permissions through a group ACL. if you\u0026rsquo;re not the user owner nor a member of the group owner and haven\u0026rsquo;t obtained any permissions through ACLs, you get the permissions of the others entity. You can quickly find files owned by a specific user or group using the find command, e.g. find / -user joeri or find / -group users\nChanging User Ownership # chown who what will change file or directory ownership, e.g. chown joeri myfile will set the user joeri as the owner of testfile.\nA particular useful option is the -R flag which allows you to change ownership recursively on a directory and everything below:\nchown -R joeri /home/joeri\nChanging Group Ownership # There are two commands to change group ownership: chown and chgrp\nchgrp users /home/account will set the group owner to users on the /home/account directory. You can use the -R option as well.\nWith the chown command there are several ways to change the group owner:\nchown joeri.users myfile\nSets the user joeri as user owner and the group users as the group owner. chown joeri:users myfile chown .users myfile chown :users myfile Again, the -R option is available.\nDefault Ownership # The user who creates a file automatically becomes the user owner, and the primary group of that user automatically becomes group owner.\nIf a user is a member of more groups, the effective primary group can be changed temporarily with the newgrp command so that new files will get a new group as group owner.\nCheck the effective primary group with the groups command. The primary group is the first group: joeri\n$ groups joeri joeri: joeri users Change the effective primary group using the newgrp command, and undo the group change using exit.\n$ newgrp users $ groups joeri joeri: users joeri $ touch file1 -rw-rw-r--. 1 joeri users 0 May 9 14:29 file1 $ exit $ groups joeri joeri: joeri users Managing Basic Permissions # Understanding Read, Write and Execute Permissions # The three basic permissions allow users to read, write and execute files. The effect of these permissions differ when applied to files or directories.\nPermission Applied to Files Applied to Directories Read Open a file List content Write Change content Create and delete files Execute Run a program Change into directory Applying Basic Permissions # You use the chmod command to apply permissions. These can be set for user, group and others in either absolute/numeric or relative mode:\nPermission Absolute Relative Read 4 R Write 2 W Execute 1 X When setting absolute permissions, you calculate the value you need. For example, setting Read+Write+Execute (4+2+1) permissions for the owner and Read+Execute (4+1) permissions for the group owner and others you would use: chown 755 somefile\nChanging permissions in absolute mode will replace all current permissions. If instead you want to modify permission relative to the current ones you work with three indicators: u, g, o for User, Group and Others.\nchmod u+rwx, g-w, o-rwx somefile would set all permissions for the user owner, remove the write permission for the group owner and remove all permissions for others.\nTo set execute permissions recursively to all directories but not to files, you can use the uppercase X: chown -R o+rX /somedirectory\nManaging Advanced Permissions # Understanding Advanced Permissions\nThere are three advanced permissions: SUID (set user id), SGID (set group id) and Sticky Bit.\nApplying the SUID permission to a file will allow a user to execute that file as if he had the user owner rights on the file. A file that has root as user ower and has the SUID permission will be executed as root by the user even if that user isn\u0026rsquo;t root at all. An example is the passwd command:\n$ ls -l /usr/bin/passwd\u0026lt;br /\u0026gt;-rwsr-xr-x. 1 root root 37600 Jan 30 02:12 /usr/bin/passwd passwd is owned by root and has the SUID permission, notice the s where the x should be in the user owner permissions.\nYou can set the SUID permission by executing chown u+s somefile\nThe second advanced permission is SGID. If applied to an executable file it gives the user who executes the file the permissions of the group owner of that file. Applied to a directory it will set the default group ownership on files and subdirectories inside that directory:\n$ mkdir somedirectory $ ls -ld somedirectory drwxrwxr-x. 2 joeri joeri 4096 May 9 16:10 somedirectory $ chown .users somedirectory $ ls -ld somedirectory drwxrwxr-x. 2 joeri users 4096 May 9 16:10 somedirectory $ chmod g+s somedirectory $ ls -ld somedirectory drwxrwsr-x. 2 joeri users 4096 May 9 16:10 somedirectory $ cd somedirectory \u0026amp;\u0026amp; touch somefile \u0026amp;\u0026amp; ls -l -rw-rw-r--. 1 joeri users 0 May 9 16:14 somefile The third advanced permission is Sticky Bit. This permission is useful to protect files against accidental deletion in an environment where multiple users have write permissions in the same directory.\nIf the Sticky Bit permission is applied to a directory, a user can delete a file only if he is the owner of the file or of the directory that contains the file.\nThe Sticky Bit permission can not be inherited from the parent directory.\nUse chmod +t followed by the name of the file or directory you want to apply the permission on.\nPermission Absolute Relative Files Directories SUID 4 u+s Execute file with the permissions of the file owner n/a SGID 2 g+s Execute file with the permissions of the group owner Files created in the directory get the same group owner. Sticky Bit 1 +t n/a Prevent users from deleting files belonging to other users. ","date":"29 May 2020","externalUrl":null,"permalink":"/basic-advanced-permissions-101/","section":"Blog","summary":"","title":"Basic \u0026 Advanced Permissions 101","type":"posts"},{"content":"","date":"29 May 2020","externalUrl":null,"permalink":"/tags/linux/","section":"Tags","summary":"","title":"Linux","type":"tags"},{"content":"","date":"29 May 2020","externalUrl":null,"permalink":"/tags/permissions/","section":"Tags","summary":"","title":"Permissions","type":"tags"},{"content":"","date":"1 May 2020","externalUrl":null,"permalink":"/tags/authenticated-origin-pulls/","section":"Tags","summary":"","title":"Authenticated Origin Pulls","type":"tags"},{"content":"","date":"1 May 2020","externalUrl":null,"permalink":"/categories/cloudflare/","section":"Categories","summary":"","title":"CloudFlare","type":"categories"},{"content":"","date":"1 May 2020","externalUrl":null,"permalink":"/tags/cloudflare/","section":"Tags","summary":"","title":"CloudFlare","type":"tags"},{"content":" In addition to my previous post on blocking requests that are hitting my websites directly without going through the CloudFlare network, we can enable the Authenticated Origin Pulls feature.\nAuthenticated Origin Pulls uses TLS Authentication to verify that the server hosting my website is communicating with CloudFlare and not some other server or client.\nNginx will be configured to only accept requests which use a valid client certificate from Cloudflare and requests which have not passed through CloudFlare will be dropped: The server will respond with a 400 Bad Request status code.\nLet\u0026rsquo;s start by downloading the CloudFlare origin pull certificate from here, and put it in an appropriate location. I\u0026rsquo;ve renamed the certificate to cloudflare.crt.\n# cd /etc/ssl/certs # wget https://support.cloudflare.com/hc/en-us/article_attachments/360044928032/origin-pull-ca.pem # mv origin-pull-ca.pem cloudflare.crt Next, we need to specify where Nginx can find this certificate in our Nginx configuration Server block. Add the following to your server block after your already existing ssl_certificate and ssl_certificate_key directives:\nssl_client_certificate /etc/ssl/certs/cloudflare.crt; ssl_verify_client on; The configuration should look similar to this:\nssl on; ssl_certificate /etc/ssl/certs/website.pem; ssl_certificate_key /etc/ssl/private/website_privatekey.pem; ssl_client_certificate /etc/ssl/certs/cloudflare.crt; ssl_verify_client on; Save the file and exit the text editor.\nTest the configuration changes by executing:\n# nginx -t If there were no problems, go ahead and apply the configuration by reloading Nginx:\n# nginx -s reload If you visit the website now, you should see the 400 Bad Request error. That means everything is working as intended and as a final step we will need to enable the Authenticated Origin Pulls feature in CloudFlare.\nOpen the SSL/TLS section in the Cloudflare dashboard, head to the Origin Server subsection and toggle the Authenticated Origin Pulls option to On.\nAuthenticated Origin Pulls on the Nginx server is now set up correctly to ensure that Nginx only accepts requests from Cloudflare’s servers, preventing anyone else from directly connecting to the Nginx server.\nOptional # While I was implementing this I decided to replace my Let\u0026rsquo;s Encrypt certificates with CloudFlare\u0026rsquo;s Origin Certificate (not origin pull certificate).\nOrigin Certificates are only valid for encryption between Cloudflare and the origin server, but that\u0026rsquo;s okay since Authenticated Origin Pulls only allows traffic going through CloudFlare to connect to my server anyway.\nYou can create or download the Origin Certificate in the same section where you enabled Authenticated Origin Pulls.\nIf you don\u0026rsquo;t have the private key, you will need to create a new certificate and either provide your own private key and CSR, or let CloudFlare generate both for you.\nOn the next screen, choose the PEM format and copy the content of the Origin Certificate section to a file called yourdomain.com.pem and the content of the private key section to a file called yourdomain.com.key.\nUpload both files to your server and place them in the following paths:\n/etc/ssl/certs/yourdomain.com.pem\n/etc/ssl/private/yourdomain.com.key\nJust like before, we need to tell Nginx where to find those files:\nssl on; ssl_certificate /etc/ssl/certs/yourdomain.com.pem; ssl_certificate_key /etc/ssl/private/yourdomain.com.key; ssl_client_certificate /etc/ssl/certs/cloudflare.crt; ssl_verify_client on; Test and reload the Nginx configuration.\n","date":"1 May 2020","externalUrl":null,"permalink":"/cloudflare-authenticated-origin-pulls/","section":"Blog","summary":"","title":"CloudFlare Authenticated Origin Pulls","type":"posts"},{"content":"","date":"1 May 2020","externalUrl":null,"permalink":"/categories/nginx/","section":"Categories","summary":"","title":"Nginx","type":"categories"},{"content":"","date":"1 May 2020","externalUrl":null,"permalink":"/categories/ssl/","section":"Categories","summary":"","title":"SSL","type":"categories"},{"content":"","date":"1 May 2020","externalUrl":null,"permalink":"/tags/ssl/","section":"Tags","summary":"","title":"SSL","type":"tags"},{"content":"","date":"1 May 2020","externalUrl":null,"permalink":"/tags/tls/","section":"Tags","summary":"","title":"TLS","type":"tags"},{"content":"","date":"1 May 2020","externalUrl":null,"permalink":"/tags/tls-authentication/","section":"Tags","summary":"","title":"TLS Authentication","type":"tags"},{"content":" My websites are behind CloudFlare, which acts as a reverse proxy and which can help in mitigating attacks, malicious traffic and requests.\nCloudFlare is masking the real IP address of this site. If you look up the DNS A record for this domain, you\u0026rsquo;ll see one of CloudFlare\u0026rsquo;s IP addresses. Essentially, CloudFlare is forwarding traffic from their servers to the server where my sites are hosted.\nThe inconvenience of this is that I can\u0026rsquo;t see the real IP address of the visitor, instead, I\u0026rsquo;m seeing CloudFlare\u0026rsquo;s IP addresses in the Nginx log.\nLuckily, Cloudflare includes the original visitor IP address in the X-Forwarded-For and CF-Connecting-IP headers. I would only need to make a simple Nginx configuration change, and, with this information, I can also block anyone who happens to know the real IP address my server and could be ypassing CloudFlare.\nInside /etc/nginx, I created an additional configuration file, cloudflare_ips.conf, where I map two variables using the ngx_http_geo_module. If the remote IP is a CloudFlare IP address, then I set it as allowed.\ngeo $remote_addr $is_allowed_ip { 173.245.48.0/20 yes; 103.21.244.0/22 yes; 103.22.200.0/22 yes; 103.31.4.0/22 yes; 141.101.64.0/18 yes; 108.162.192.0/18 yes; 190.93.240.0/20 yes; 188.114.96.0/20 yes; 197.234.240.0/22 yes; 198.41.128.0/17 yes; 162.158.0.0/15 yes; 104.16.0.0/12 yes; 172.64.0.0/13 yes; 131.0.72.0/22 yes; 2400:cb00::/32 yes; 2606:4700::/32 yes; 2803:f800::/32 yes; 2405:b500::/32 yes; 2405:8100::/32 yes; 2a06:98c0::/29 yes; 2c0f:f248::/32 yes; default no; } You can then add the following if block into the site\u0026rsquo;s server block configuration. Basically, if the remote IP is not in the list above, the $is_allowed_ip variable will be set to no and a 403 Forbidden status code is returned.\nif ($is_allowed_ip = no ){ return 403; } We shouldn\u0026rsquo;t forget the include the cloudflare_ips.conf file inside our main nginx.conf. Add the below into the http block.\ninclude /etc/nginx/cloudflare_ips.conf; The last step would be to modify the existing log format, so that it includes the real ip address of the visitor, in addition to the CloudFlare IP. The log format can be found in nginx.conf. You would need to add the$http_x_forwarded_for variable in the log_format directive.\nTest the Nginx config and reload Nginx to apply the above changes.\n","date":"16 July 2019","externalUrl":null,"permalink":"/blocking-requests-not-originating-from-cloudflare-on-nginx/","section":"Blog","summary":"","title":"Blocking requests not originating from CloudFlare on Nginx","type":"posts"},{"content":" An SSL Certificate is a text file with encrypted data that you install on your server so that you can secure/encrypt sensitive communications between your site and your visitors. They are also used to verify that you are connected with the service you wish to be connecting with, and, as a website owner it validates your trustworthiness.\nSSL certificates can be expensive, so here\u0026rsquo;s where Let’s Encrypt comes into play.\nLet’s Encrypt is a free, automated, and open certificate authority (CA), run for the public’s benefit. It is a service provided by the Internet Security Research Group (ISRG: We give people the digital certificates they need in order to enable HTTPS (SSL/TLS) for websites, for free, in the most user-friendly way we can. We do this because we want to create a more secure and privacy-respecting Web.\nIn order to have Let\u0026rsquo;s Encrypt issue a valid certificate for your site, it needs to validate your domain. In other words, it needs to make sure that whoever is requesting the certificate is also in full control of the domain name it\u0026rsquo;s being issued for.\nThere are different ways you can prove you\u0026rsquo;re the owner of a specific domain:\nProvisioning a DNS record under example.com, or Provisioning an HTTP resource under a well-known URI on http://example.com/ The first method is done by manually adding a specific DNS record to your domain. The latter is done automatically by the Let\u0026rsquo;s Encrypt agent on your server.\nIn this guide, we\u0026rsquo;ll be using the second method with certbot to install an SSL certificate in a matter of minutes. This method requires that your domain is pointing to the server you\u0026rsquo;re running certbot on with its DNS.\nCertbot is a free, open source software tool for automatically using Let’s Encrypt certificates on manually-administrated websites to enable HTTPS.\nFirst, let\u0026rsquo;s download certbot, copy it into our PATH and apply the necessary permissions:\n# wget https://dl.eff.org/certbot-auto # mv certbot-auto /usr/local/bin/certbot-auto # chown root /usr/local/bin/certbot-auto # chmod 0755 /usr/local/bin/certbot-auto When you run certbot-auto --nginx, certbot will look into your Nginx configuration files for domains it can request an SSL certificate for. You\u0026rsquo;ll be presented with a menu:\nEnter the numbers of the domains you wish to generate an SSL certificate for, separated by commas or spaces and hit ENTER. Or leave the input blank to select all domains. Choose if you wish to have the webserver enforce HTTPS traffic (option 2) or not (option 1).\nYou now have a valid SSL certificate on for your domain:\nCheck your Nginx configuration to see what exactly was added by Certbot.\n","date":"31 May 2019","externalUrl":null,"permalink":"/automate-lets-encrypt-ssl-with-certbot-on-centos-8/","section":"Blog","summary":"","title":"Automate Let’s Encrypt SSL with Certbot on Centos 8","type":"posts"},{"content":"","date":"31 May 2019","externalUrl":null,"permalink":"/tags/centos/","section":"Tags","summary":"","title":"CentOS","type":"tags"},{"content":"","date":"1 May 2019","externalUrl":null,"permalink":"/tags/gunicorn/","section":"Tags","summary":"","title":"Gunicorn","type":"tags"},{"content":"","date":"1 May 2019","externalUrl":null,"permalink":"/categories/python/","section":"Categories","summary":"","title":"Python","type":"categories"},{"content":"","date":"1 May 2019","externalUrl":null,"permalink":"/tags/python/","section":"Tags","summary":"","title":"Python","type":"tags"},{"content":" This post explains how to serve a Python app from a Virtualenv with the Gunicorn WSGI and using Nginx as a proxy server.\nYou should already have Nginx installed, and have a sample Flask app in a Virtualenv.\nInstall Gunicorn in your Virtualenv # Activate your Virtualenv and install Gunicorn by typing:\n$ source myprojectenv/bin/activate (myprojectenv)$ pip install gunicorn Creating the WSGI Entry Point # Next, we\u0026rsquo;ll create a file called wsgi.py that will serve as the entry point for our application code in app.py.\n(myprojectenv)$ nano wsgi.py We\u0026rsquo;ll import the Flask application code from app.py inside the entry point file:\nfrom app import app if name == \u0026#34;main\u0026#34;: app.run() Save and close the file.\nTesting Gunicorn # We should check that Gunicorn can serve the application correctly.\nWe can do this by simply passing the gunicorn command the name of our entry point. This is the name of the entry point file (minus the .py extension), plus the name of the application. In this case, this is wsgi:app.\nWe’ll also specify a publicly available interface and port to bind to:\n(myprojectenv)$ gunicorn --bind 0.0.0.0:5000 wsgi:app You\u0026rsquo;ll see similar output as:\nJan 03 22:13:26 odin gunicorn[347]: [2020-01-03 22:13:26 +0400] [347] [INFO] Starting gunicorn 19.9.0 Jan 03 22:13:26 odin gunicorn[347]: [2020-01-03 22:13:26 +0400] [347] [\u0026lt;code\u0026gt;INFO] Listening at: http://0.0.0.0:5000 (28217) Jan 03 22:13:26 odin gunicorn[347]: [2020-01-03 22:13:26 +0400] [347] [INFO] Using worker: sync Jan 03 22:13:26 odin gunicorn[347]: [2020-01-03 22:13:26 +0400] [367] [INFO] Booting worker with pid: 367 Visit your server’s IP address with :5000 appended to the end in your web browser to see your application:\nhttp://your_server_ip:5000 When you have confirmed that it’s functioning properly, press CTRL-C in your terminal window and deactivate the virtual environment.\n(myprojectenv)$ deactivate Creating the Systemd Script # A Systemd service unit file will allow the OS\u0026rsquo;s init system to automatically start Gunicorn and serve the Flask application whenever the server boots.\nLet\u0026rsquo;s begin by creating a service unit file within the /etc/systemd/system directory:\n$ sudo nano /etc/systemd/system/myproject.service We\u0026rsquo;ll add the [Unit] section which contains a description of the service and we\u0026rsquo;ll only allow this service to start if required network services are running as well.\n[Unit] Description=Gunicorn instance to serve myproject After=network.target In the [Service] section we\u0026rsquo;ll specify the user and group we want our process to be running under. I\u0026rsquo;m specifying my own username since it owns all of the files. So that Nginx can communicate with the Gunicorn processes, we\u0026rsquo;ll give group ownership to the www-data group.\n[Service] User=joeri Group=www-data Next, let’s map out the working directory and set the PATH environmental variable so that the init system knows that the executables for the process are located within our virtual environment. Let’s also specify the command to start the service. This command will do the following:\nStart 3 worker processes Create and bind to a Unix socket file, myproject.sock, within our project directory. We’ll set an umask value of 007 so that the socket file is created giving access to the owner and group, while restricting access to others. Specify the WSGI entry point file name, along with the Python callable within that file (wsgi:app) Systemd requires that we give the full path to the Gunicorn executable, which is installed within our virtual environment.\nWorkingDirectory=/home/joeri/www/myproject Environment=\u0026#34;PATH=/home/joeri/www/myproject/myprojectenv/bin\u0026#34; ExecStart=/home/joeri/www/myproject/myprojectenv/bin/gunicorn --workers 3 --bind unix:myproject.sock -m 007 wsgi:app Next, we want this service to start when the system boots up in multi-user mode with networking: multi-user.target\nWe do this by adding the [Install] section and specifying the target.\n[Install] WantedBy=multi-user.target Our complete service unit file should look like the below:\n[Unit] Description=Gunicorn instance to serve myproject After=network.target [Service] User=joeri Group=www-data WorkingDirectory=/home/joeri/www/myproject Environment=\u0026#34;PATH=/home/joeri/www/myproject/myprojectenv/bin\u0026#34; ExecStart=/home/joeri/www/myproject/myprojectenv/bin/gunicorn --workers 3 --bind unix:myproject.sock -m 007 wsgi:app [Install] WantedBy=multi-user.target We can now start our service and enable it at boot time:\n$ sudo systemctl start myproject $ sudo systemctl enable myproject Proxy requests with Nginx # We can now configure Nginx to pass web requests to that socket by making some small additions to the nginx configuration file.\nVerify if the /etc/nginx/proxy_params file exists. If not, create it with the following content:\nproxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 100M; client_body_buffer_size 1m; proxy_intercept_errors on; proxy_buffering on; proxy_buffer_size 128k; proxy_buffers 256 16k; proxy_busy_buffers_size 256k; proxy_temp_file_write_size 256k; proxy_max_temp_file_size 0; proxy_read_timeout 300; Create a new Nginx configuration file for your application and add the below lines to it:\nsudo nano /etc/nginx/sites-available/myproject.conf server { listen 80; server_name your_domain www.your_domain; location / { include proxy_params; proxy_pass http://unix:/home/joeri/www/myproject/myproject.sock; } } Save and close the file when you’re finished.\nTo enable the Nginx server block configuration you’ve just created, link the file to the sites-enabled directory:\nsudo ln -s /etc/nginx/sites-available/myproject /etc/nginx/sites-enabled With the file in that directory, you can test for syntax errors:\n$ sudo nginx -t If no errors are returned, restart the Nginx process to read the new configuration:\n$ sudo systemctl restart nginx You should now be able to navigate to your server’s domain name in your web browser and see your application\u0026rsquo;s output. In the next post I\u0026rsquo;ll set up SSL for this application.\n","date":"1 May 2019","externalUrl":null,"permalink":"/serving-a-python-flask-app-with-gunicorn-nginx-and-systemd/","section":"Blog","summary":"","title":"Serving a Python Flask app with Gunicorn, Nginx and Systemd.","type":"posts"},{"content":"","date":"1 May 2019","externalUrl":null,"permalink":"/categories/web/","section":"Categories","summary":"","title":"Web","type":"categories"},{"content":" In this post, I\u0026rsquo;ll walk through how to install Nginx and set up Nginx server blocks on CentOS 8, and, how to serve different content to different visitors depending on which domains they are requesting.\nAfter initially installing CentOS, check if updates are available and apply them:\n$ sudo dnf update -y firewalld # Next, let\u0026rsquo;s install the firewalld daemon to secure the server, start the daemon and enable it at boot. We\u0026rsquo;ll also configure the firewall daemon to only allow requests over the http and https protocols.\n$ sudo dnf install -y firewalld $ sudo systemctl start firewalld $ sudo systemctl enable firewalld $ sudo firewall-cmd --zone=public --permanent --add-service=http $ sudo firewall-cmd --zone=public --permanent --add-service=https $ sudo firewall-cmd --reload Install Nginx # We\u0026rsquo;ll proceed with installing Nginx, starting the server and enabling the service at boot:\n$ sudo dnf install -y nginx $ sudo systemctl start nginx $ sudo systemctl enable nginx We can run the following cURL command to test the service:\n$ curl http://localhost You should see the HTML code of the Nginx test page, confirming that the service is running.\nCreating directory structures for different websites # The example configuration in this guide will make one server block for example.com and another for example2.com.\nWe\u0026rsquo;ll configure DNS for these dummy domains locally in the /etc/hosts file.\nFirst, we need to make a directory structure that will hold the site data to serve to visitors:\n$ sudo mkdir -p /var/www/example.com $ sudo mkdir /var/www/example2.com We now need to modify permissions on these directories so our regular user can make modifications to the files inside, and so that Nginx can read the files as well.\n$ sudo chown -R joeri:nginx /var/www/example.com $ sudo chown -R joeri:nginx /var/www/example2.com We\u0026rsquo;ll also ensure read access for the user and group to the /var/www/ directory and all files and directories inside:\n$ sudo chown -R 755 /var/www Creating Demo Pages # We need to create some content for Nginx to serve to visitors. We can do that by creating a simple HTML file inside the directories we previously created. As a regular user, create the HTML file and add your content:\n$ nano /var/www/example.com/index.html \u0026lt;html\u0026gt; \u0026lt;body\u0026gt; \u0026lt;h1\u0026gt;Welcome To Example.com\u0026lt;/h1\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; Save and exit, then do the same for /var/www/example2.com, replacing the content of the h1 element with “Welcome To Example2.com”.\nCreating \u0026amp; Enabling the Server Block Files # Server block files specify the configuration of our separate sites and tell the Nginx web server how to respond to various domain requests.\nI\u0026rsquo;ll start by replicating the Ubuntu/Debian directory structure for Nginx server block files since I really like that method.\n$ sudo mkdir /etc/nginx/sites-available $ sudo mkdir /etc/nginx/sites-enabled Next, we tell Nginx to look for server blocks in the sites-enabled directory. Add the following line to the end of the http {} block in /etc/nginx/nginx.conf:\ninclude /etc/nginx/sites-enabled/*.conf; Create the server block for the example.com site in the /etc/nginx/sites-available/example.com.conf file:\n$ sudo nano /etc/nginx/sites-available/example.com.conf Add the following lines to the file:\nserver { listen 80; server_name example.com www.example.com; location / { root /var/www/example.com/html; index index.html index.htm; try_files $uri $uri/ =404; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } } Do the same for the example2.com site, replacing example.com with example2.com.\nWe can now enable both server block files by creating a symlink to the /etc/nginx/sites-enabled directory:\n$ sudo ln -s /etc/nginx/sites-available/example.com.conf /etc/nginx/sites-enabled/example.com.conf $ sudo ln -s /etc/nginx/sites-available/example2.com.conf /etc/nginx/sites-enabled/example2.com.conf Test and restart Nginx to make the changes take effect.\n$ sudo nginx -t $ sudo systemctl restart nginx Creating DNS entries # We have two dummy domains, but they aren\u0026rsquo;t pointing to our Nginx server with their DNS. You can create a local DNS entry for those domains in /etc/hosts. I\u0026rsquo;m doing this locally on the machine that\u0026rsquo;s running the webserver:\n$ sudo nano /etc/hosts 127.0.0.1 localhost example.com www.example.com example2.com www.example2.com On a remote (Linux) machine you would add an additional line, starting with the IP address of your server:\n192.168.100.99 example.com www.example.com example2.com www.example2.com Our machine can now resolve those domains to a specific server.\nTesting results # Now that DNS is in place, we can run some tests. Either with cURL or by opening the domains in your browser.\n$ curl http://example.com $ curl http://example2.com You should see different outputs depending on the domain that was called.\nIf you keep seeing a 403 Forbidden error, and, you have SELinux enabled, you will want to relabel the /var/www/ directory with the appropriate SELinux context label:\n$ sudo restorecon -Rv /var/www/ ","date":"16 April 2019","externalUrl":null,"permalink":"/installing-nginx-and-setting-up-server-blocks-on-centos-8/","section":"Blog","summary":"","title":"Installing Nginx and Setting up Server Blocks on CentOS 8","type":"posts"},{"content":" I\u0026rsquo;m dual booting between RHEL 8 and Windows 10 and the NTFS drive I use to share data between my two operating systems was suddenly mounted Read Only on Linux.\nWhile trying to mount the drive again, I was facing the following message:\n[root@rhel8 mnt]# umount -l /dev/sda1 [root@rhel8 mnt]# mount -t ntfs-3g -o rw /dev/sda1 /mnt/data The disk contains an unclean file system (0, 0). Metadata kept in Windows cache, refused to mount. Falling back to read-only mount because the NTFS partition is in an unsafe state. Please resume and shutdown Windows fully (no hibernation or fast restarting.) I don\u0026rsquo;t use hibernation in Windows and fast restarting is definitely turned off. I didn\u0026rsquo;t put Windows to sleep either, so I\u0026rsquo;m not sure where this is coming from at this point, but since I needed to write to the disk I needed an immediate fix without rebooting.\nTo fix the issue, I need the ntfsfix application:\n[root@rhel8 ~]# dnf whatprovides ntfsfix Updating Subscription Management repositories. ntfsprogs-2:2017.3.23-11.el8.x86_64 : NTFS filesystem libraries and utilities Repo : epel Matched from: Filename : /usr/bin/ntfsfix [root@rhel8 ~]# dnf install -y ntfsprogs Then I ran ntfsfix against /dev/sda1:\n[root@rhel8 ~]# ntfsfix /dev/sda1 Mounting volume... The disk contains an unclean file system (0, 0). Metadata kept in Windows cache, refused to mount. FAILED Attempting to correct errors... Processing $MFT and $MFTMirr... Reading $MFT... OK Reading $MFTMirr... OK Comparing $MFTMirr to $MFT... OK Processing of $MFT and $MFTMirr completed successfully. Setting required flags on partition... OK Going to empty the journal ($LogFile)... OK Checking the alternate boot sector... OK NTFS volume version is 3.1. NTFS partition /dev/sda1 was processed successfully. That\u0026rsquo;s it, the system then mounted the disk according to the options I\u0026rsquo;ve specified in /etc/fstab and I can now write again to my NTFS disk.\n","date":"14 February 2019","externalUrl":null,"permalink":"/mounting-an-unclean-ntfs-file-system/","section":"Blog","summary":"","title":"Mounting an unclean NTFS file system","type":"posts"},{"content":"","date":"14 February 2019","externalUrl":null,"permalink":"/tags/ntfs/","section":"Tags","summary":"","title":"Ntfs","type":"tags"},{"content":"","date":"14 February 2019","externalUrl":null,"permalink":"/tags/unclean/","section":"Tags","summary":"","title":"Unclean","type":"tags"},{"content":"","date":"15 January 2019","externalUrl":null,"permalink":"/tags/dual-display/","section":"Tags","summary":"","title":"Dual Display","type":"tags"},{"content":"","date":"15 January 2019","externalUrl":null,"permalink":"/tags/external-monitor/","section":"Tags","summary":"","title":"External Monitor","type":"tags"},{"content":"","date":"15 January 2019","externalUrl":null,"permalink":"/tags/nvidia/","section":"Tags","summary":"","title":"Nvidia","type":"tags"},{"content":" Yesterday I decided to go ahead and install RHEL8 on my laptop and use it as my everyday workstation as well as a learning platform to move forward on my Red Hat certification path.\nIn case you\u0026rsquo;re interested, you can actually use RHEL for free with a No-Cost RHEL Developer Subscription as long as you don\u0026rsquo;t use the machine \u0026ldquo;in production\u0026rdquo;.\nAfter a pretty straightforward installation of RHEL 8 I was immediately confronted with a first issue. My external HDMI monitor wasn\u0026rsquo;t detected.\nMy laptop uses the Nvidia Optimus technology to seamlessly switch between the integrated Intel GPU and the Nvidia discrete graphics. I couldn\u0026rsquo;t get it to work with the default opensource nouveau drivers, so I went ahead and installed the non-free Nvidia proprietary drivers.\nBy default, RHEL installs the (newer) Wayland display server, but the Nvidia drivers don\u0026rsquo;t seem to work with those so I had to switch to Xorg. Either way, I didn\u0026rsquo;t mind since the Nvidia drivers offer better performance.\nInstalling the non-free Nvidia drivers on RHEL 8 # The nouveau kernel module should be blacklisted first, to prevent it from loading during the next boot:\n# echo \u0026#39;blacklist nouveau\u0026#39; \u0026gt;\u0026gt; /etc/modprobe.d/blacklist.conf The next step will be to install the Xorg display server and the kernel development tools:\n# dnf groupinstall \u0026#34;base-x\u0026#34; \u0026#34;Legacy X Window System Compatibility\u0026#34; \u0026#34;Development Tools\u0026#34; # dnf install elfutils-libelf-devel \u0026#34;kernel-devel-uname-r == $(uname -r)\u0026#34; Backup and rebuild the initramfs:\n# mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r)-nouveau.img # dracut -f Make sure you have Dynamic Kernel Module Support installed. This will automatically rebuild the Nvidia kernel modules when you update the kernel, so you won\u0026rsquo;t have to reinstall them afterward.\n# dnf install dkms Download the Nvidia drivers from https://www.nvidia.com/\nChange the default runlevel:\n# systemctl set-default multi-user.target Reboot your system and install the driver:\n# chmod +x NVIDIA-$version.run # ./NVIDIA-$version.run Test the new driver by switching to the graphical.target runlevel:\n# systemctl isolate graphical.target Correct the default runlevel:\n# systemctl set-default graphical.target Configuring Xorg # First, search for your hardware and take note of the PCI bus it\u0026rsquo;s operating on. The PCI bus for my Intel GPU is 00:02.0 and for my Nvidia GPU 01:00.0.\n# lspci | grep -EA1 \u0026#39;VGA|3D\u0026#39; 00:02.0 VGA compatible controller: Intel Corporation UHD Graphics 630 (Mobile) 01:00.0 3D controller: NVIDIA Corporation GP107M \u0026amp;#91;GeForce GTX 1050 Ti Mobile] (rev a1) Next, if you have an existing Xorg configuration (/etc/X11/xorg.conf), make a backup, then replace the existing config with the below. Make sure you modify the BusID\u0026rsquo;s.\nSection \u0026#34;ServerLayout\u0026#34; Identifier \u0026#34;layout\u0026#34; Screen 0 \u0026#34;nvidia\u0026#34; Inactive \u0026#34;intel\u0026#34; EndSection Section \u0026#34;Device\u0026#34; Identifier \u0026#34;nvidia\u0026#34; Driver \u0026#34;nvidia\u0026#34; BusID \u0026#34;PCI:01:00:0\u0026#34; EndSection Section \u0026#34;Screen\u0026#34; Identifier \u0026#34;nvidia\u0026#34; Device \u0026#34;nvidia\u0026#34; Option \u0026#34;AllowEmptyInitialConfiguration\u0026#34; EndSection Section \u0026#34;Device\u0026#34; Identifier \u0026#34;intel\u0026#34; Driver \u0026#34;modesetting\u0026#34; BusID \u0026#34;PCI:00:2:0\u0026#34; EndSection Section \u0026#34;Screen\u0026#34; Identifier \u0026#34;intel\u0026#34; Device \u0026#34;intel\u0026#34; EndSection The last steps consist of autostarting the dual-display configuration using xrandr. Execute nano /etc/xdg/autostart/nvidia-optimus.desktop and add the following lines :\n[Desktop Entry] Type=Application Name=NVIDIA Optimus Exec=sh -c \u0026#34;xrandr --setprovideroutputsource modesetting NVIDIA-0; xrandr --auto\u0026#34; NoDisplay=true X-GNOME-Autostart-Phase=DisplayServer Copy the same file to /usr/share/gdm/greeter/autostart/nvidia-optimus.desktop.\nYou should now have a working dual display after logging out and logging back in or rebooting!\n","date":"15 January 2019","externalUrl":null,"permalink":"/red-hat-enterprise-linux-8-and-nvidia-optimus/","section":"Blog","summary":"","title":"Red Hat Enterprise Linux 8 and Nvidia Optimus","type":"posts"},{"content":"","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"}]