Kubernetes basics

Preface: This article will introduce the important concept in the foundation of Kubernetes - Pod, as well as its application and functions in Kubernetes. First, we will explain in detail the basic concept and composition of Pod, including its characteristics as the smallest deployable unit and the resources it contains. Then, we will explore the role and functions of Pod in Kubernetes, including scheduling unit, network unit, storage unit, life cycle management and horizontal expansion. By gaining a deeper understanding of Pods, you will gain a more comprehensive understanding of the core concepts in Kubernetes.

I. Kubernetes basics: Pod understanding and application

1.1 Basic concept and composition of Pod

In Kubernetes (K8s for short), Pod (container group) is the smallest deployable unit. It is a logical host for a set of one or more containers that can run in a Kubernetes cluster. Pods provide a self-contained environment that contains all the resources needed to run an application, such as storage, networking, and other dependencies.

Pods typically consist of one or more closely related containers that share the same namespace, network, and storage volumes. They can communicate through localhost on the local host and can share part or all of the file system.

1.2 The role and function of Pod in Kubernetes

The role of Pod in Kubernetes is as follows:

Scheduling unit: Kubernetes uses Pod as the basic unit of scheduling and determines which node to run the Pod on.
Network unit: Each Pod has its own IP address and can communicate with other Pods or external services through service discovery mechanisms inside and outside the Kubernetes cluster.
Storage unit: Pods can share storage volumes, and containers can share data in the file system.
Lifecycle management: Pods can be created, started, stopped, and destroyed, and their lifecycle is managed by the Kubernetes controller.
Horizontal expansion: Application instances can be horizontally expanded by replicating Pods.

II. Kubernetes application practice: Installing and configuring Miniconda in Kubernetes

2.1 Steps to install and configure Miniconda

This article explains how to install and configure Miniconda in a Kubernetes cluster. Miniconda is a lightweight Python environment management tool that can be used to create and manage Python environments and their related packages.

Step 1: Log in to the terminal of the Kubernetes Pod

1	kubectl exec -it <pod-name> -- /bin/bash

will <pod-name> Replace with the name of the Pod you want to log in to.

Step 2: Download and install Miniconda

1 2	curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh bash Miniconda3-latest-Linux-x86_64.sh

This will download Miniconda’s installation script and start the installation process.

Step 3: Complete the installation wizard

According to the prompts of the installation wizard, select the installation path, environment variable configuration and other options to complete the installation of Miniconda.

Step 4: Activate Miniconda environment

1	source ~/.bashrc

Reload the terminal or execute the above command to activate the Miniconda environment.

Step 5: Use Miniconda

In an activated Miniconda environment, you can use the following commands to manage the environment and install Python packages:

1
2
3

conda create --name myenv python=3.9
conda activate myenv
conda install package_name

2.2 Precautions for installing and using Miniconda

Please adjust Miniconda's download link and installation command according to the Pod's operating system and architecture.
To automate the installation and configuration of Miniconda, include the relevant steps and environment configuration in the Pod's initialization script or container image building process.
Please follow appropriate best practices and security measures when installing and using Miniconda, and configure and manage it according to your specific needs.

2.3 Summary of installing and configuring Miniconda

By following the steps in this article to install and configure Miniconda in Kubernetes, you can easily manage your Python environment and packages and provide the required dependencies for your application. Miniconda's flexibility and scalability make it ideal for developing and deploying Python applications in Kubernetes environments.

III. Kubernetes data operation: process of transferring files to Kubernetes cluster on local Mac computer

This article describes how to transfer files to Pods in a Kubernetes cluster from your local Mac computer. We use the lrzsz tool to implement file upload and download operations.

3.1 Preparations before file transfer

Homebrew is already installed on your local Mac computer.
The Kubernetes cluster already has the lrzsz tool installed.

[Execute locally]

Open the Terminal application.
Install the lrzsz tool. Execute the following command in the terminal:
1
brew install lrzsz
Make sure the lrzsz tool is installed in the Kubernetes cluster. Execute the following command in the terminal in the Kubernetes cluster:
1
2
apt-get update
apt-get install lrzsz

[Executed in a Kubernetes cluster]

Log in to the Pod's terminal. Execute the following command in the terminal in the Kubernetes cluster:
1
kubectl exec -it <pod-name> -- /bin/bash
will <pod-name> Replace with the name of the target Pod.
In the Pod's terminal, use the following command to receive the file:
1
rz
After executing this command, a file selection window will pop up.

3.2 Specific steps for file transfer

[Execute locally]

In a local terminal, use the following command to send the file to the Kubernetes cluster’s Pods:
will /path/to/environ.yaml Replace with environ.yaml The path of the file on the local computer.
1
sz /path/to/environ.yaml

[Executed in Kubernetes cluster]

In the file selection window of the terminal in the Kubernetes cluster, select the file to upload environ.yaml。

File transfers will be performed between local and Kubernetes clusters using the upload and download commands of the lrzsz tool. Please make sure to follow the appropriate steps to install and operate both locally and in the Kubernetes cluster, and use the correct commands for file transfers.

Please follow appropriate security and best practices when performing file transfer operations to protect the security of your data and systems.

IV. How to check if Linux commands continue to execute after the terminal is disconnected

When running a command or script in a Linux terminal, if the terminal connection is lost, you may wonder if the command or script is still executing in the background. Here are a few ways to check if Linux commands continue to execute after the terminal is disconnected.

4.1 Method 1: Use ps command

Open a new terminal window.
Run the following command:ps aux | grep <命令或脚本关键词>
Check the output for any processes related to the command or script. If present, the command or script is still executing in the background.

4.2 Method 2: Use pgrep command

Open a new terminal window.
Run the following command:pgrep -f <命令或脚本关键词>
Check the output for a process ID associated with the command or script. If present, the command or script is still executing in the background.

4.3 Method 3: Use log files or output files

If output redirection is used in a command or script (e.g. tee), please check the log file or output file.
Open a new terminal window.
Use tail Command to view the last few lines of a log file or output file:tail -n <行数> <文件路径>
Check whether the last few lines contain anything relevant to the output of the command or script. If there is new output, it means the command or script is still executing.

It's important to note that even if commands or scripts are still executing in the background after the terminal is disconnected, they may terminate or stop running if an error or problem occurs. Therefore, you should also check for problems with the command or script itself.

Summary:
You can check whether Linux commands continue to execute after the terminal is disconnected by using the ps command, the pgrep command, or viewing the log file or output file. These methods provide a way to know whether a command or script is continuing to execute in the background to ensure that the task is functioning properly.

V. Execute Python script using Bash script

This document describes how to use a Bash script to execute a specified Python script and provides an example script. The script also involves using the conda environment to run Python.

5.1 Introduction

In some cases, you may need to execute a long-running Python script in the terminal. To ensure persistence and ease management, you can write a Bash script to run the Python script. This document provides a sample script that demonstrates how to use a Bash script to execute a Python script.

5. 2 Script Example

The following is an example Bash script that executes a specific Python script, using the conda environment:

#!/bin/bash

# 定义 conda 环境名称
conda_env="python3.10"

# 激活 conda 环境
source activate $conda_env

# 执行 Python 脚本，并使用 tee 将输出同时重定向到文件和控制台
python data_gen_updated.py conversations_0607_v1_500 500 | tee output.log

# 停用 conda 环境
conda deactivate

The above script contains the following steps:

Define the name of the conda environment (modify as needed).
Use source activate The command activates the specified conda environment.
Use python Command executes a specific Python script. At the same time, use tee The command redirects output to both a file and the console.
Use conda deactivate command to deactivate the conda environment.

5.3 Using scripts

Follow these steps to use the script in the terminal:

Use a text editor to create a new file and paste the above example script into it.
Save the file and close the text editor.
In the terminal, give the script execution permissions:
1
chmod +x script.sh
Run the script:
1
./script.sh

Make sure that the required conda environment is properly installed and configured before running the script, and change the python data_gen_updated.py conversations_0607_v1_500 500 Replace with the actual command you want to execute.

5.4 Conclusion

Use Bash scripts to execute Python scripts in the terminal and provide persistence and management flexibility. This document provides a sample script to get you started using Bash scripts to execute Python scripts, and demonstrates using the conda environment. The feet can be modified and adjusted according to your actual needs