Technical Discussion: High Availability with Kolla-Ansible

In our past technical practices, we have discussed how to achieve containerized deployment of the OpenStack cloud platform using the Kolla-Ansible tool. Now, with the dual demands of enterprise users for operational experience and system resilience, this article will integrate the new Skyline component into the Kolla-Ansible toolchain.

Skyline, as the next-generation dashboard for OpenStack, features a more modern web design, optimizing user experience and enhancing functionality. This article also integrates high availability configuration for Kolla-Ansible, making OpenStack more secure and reliable.

1. Environment Preparation

1.1 Role Planning

Node Name IP Address Role
controller1 ens192: 192.168.1.1 ens224: Control Node, Network Node, Deployment Node
controller2 ens192: 192.168.1.2 ens224: Control Node, Network Node
controller3 ens192: 192.168.1.3 ens224: Control Node, Network Node
compute1 ens192: 192.168.1.4 ens224: Compute Node

1.2 Environment Preparation

Network Card Configuration

cat > /etc/sysconfig/network-scripts/ifcfg-ens224 <<EOF
NAME=ens224
DEVICE=ens224
TYPE=Ethernet
ONBOOT="yes"
BOOTPROTO="none"
EOF
# Reload ens34 network card device
nmcli con reload && nmcli con up ens224

Set Hostname

hostnamectl set-hostname controller1 && bash
hostnamectl set-hostname controller2 && bash
hostnamectl set-hostname controller3 && bash

Configure Cluster SSH Trust Relationship

ssh-keygen
ssh-copy-id 192.168.1.1
ssh-copy-id 192.168.1.2
ssh-copy-id 192.168.1.3

Kernel Parameter Optimization

echo 'modprobe br_netfilter' >> /etc/rc.d/rc.local
chmod 755 /etc/rc.d/rc.local
modprobe br_netfilter
echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
echo 'net.bridge.bridge-nf-call-iptables=1' >> /etc/sysctl.conf
echo 'net.bridge.bridge-nf-call-ip6tables=1'  >>/etc/sysctl.conf
sysctl -p

2. Deploy Kolla Ansible

Install Dependencies

yum install -y python-devel libffi-devel gcc openssl-devel libselinux-python python2-pip  python-pbr epel-release ansible

Configure Aliyun pip source

mkdir ~/.pip
cat > ~/.pip/pip.conf << EOF
[global]
trusted-host=mirrors.aliyun.com
index-url=https://mirrors.aliyun.com/pypi/simple/
EOF

Install Kolla-Ansible

OpenStack and Kolla version correspondence:

https://releases.openstack.org/teams/kolla.html

pip install setuptools==22.0.5
pip install pip==20.3.4
pip install wheel
pip install kolla-ansible==9.1.0 --ignore-installed PyYAML
mkdir -p /etc/kolla
chown $USER:$USER /etc/kolla
cp -r /usr/share/kolla-ansible/etc_examples/kolla/* /etc/kolla
cp /usr/share/kolla-ansible/ansible/inventory/* .
cat << EOF | sed -i '/^\[defaults\]$/ r /dev/stdin' /etc/ansible/ansible.cfg
host_key_checking=False
pipelining=True
forks=100
EOF

Configure Host Inventory File Based on Deployment Scenario

[root@controller1 ~]# cat multinode
# These initial groups are the only groups required to be modified.
# The additional groups are for more control of the environment.
[control]
# These hostname must be resolvable from your deployment host
controller1
controller2
controller3
# The above can also be specified as follows:
#control[01:03]     ansible_user=kolla
# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
controller1
controller2
controller3
[compute]
compute1
[monitoring]
controller1
controller2
controller3
# When compute nodes and control nodes use different interfaces,
# you need to comment out "api_interface" and other interfaces from the globals.yml
# and specify like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1
[storage]
[deployment]
localhost       ansible_connection=local...

Check if inventory configuration is correct

ansible -i multinode all -m ping

Technical Discussion: High Availability with Kolla-Ansible

Generate Kolla Passwords

kolla-genpwd
# Modify keystone_admin_password to customize, it will be used when logging into the dashboard
sed -i 's#keystone_admin_password:.*#keystone_admin_password: kolla#g' /etc/kolla/passwords.yml
cat /etc/kolla/passwords.yml | grep keystone_admin_password

Configure global.yml

[root@controller1 ~]# cat /etc/kolla/globals.yml | grep -v '^#'| grep -v '^$'
---
kolla_internal_vip_address: "192.168.1.100"
network_interface: "ens192"
kolla_external_vip_interface: "{{ network_interface }}"
api_interface: "{{ network_interface }}"
storage_interface: "{{ network_interface }}"
cluster_interface: "{{ network_interface }}"
swift_storage_interface: "{{ storage_interface }}"
swift_replication_interface: "{{ swift_storage_interface }}"
tunnel_interface: "{{ network_interface }}"
dns_interface: "{{ network_interface }}"
neutron_external_interface: "ens224"
neutron_plugin_agent: "openvswitch"
keepalived_virtual_router_id: "66"
kolla_base_distro: "centos"
kolla_install_type: "source"
openstack_release: "train"
enable_neutron_provider_networks: "yes"
docker_registry: "kolla-register.ssturing.com"
docker_namespace: "kolla"
enable_grafana: "yes"
enable_prometheus: "yes"
enable_neutron_vpnaas: "yes"
enable_neutron_fwaas: "yes"
enable_neutron_qos: "yes"
enable_neutron_agent_ha: "yes"
# Parameter description:
kolla_internal_vip_address: Virtual IP (VIP) for high availability of control nodes, used for OpenStack API service high availability
keepalived_virtual_router_id: Virtual router ID for Keepalived, used for VIP heartbeat detection
enable_neutron_agent_ha: Enable Neutron Agent high availability to prevent single point of failure in network services
openstack_release: Specify the OpenStack version to be deployed as Train
neutron_plugin_agent: Use Open vSwitch as the underlying network plugin for Neutron
enable_neutron_provider_networks: Support Provider Networks (e.g., VLAN, Flat networks)
enable_neutron_vpnaas, enable_neutron_fwaas, enable_neutron_qos: Enable advanced features such as VPN as a service, firewall as a service, and quality of service
enable_grafana, enable_prometheus: Integrate Grafana and Prometheus for monitoring visualization
docker_registry: Specify the private Docker image repository address for pulling Kolla images
docker_namespace: Namespace for the Docker image repository

Deploy OpenStack

# Pre-configuration, install docker, docker sdk, disable firewall, configure time synchronization, etc.
kolla-ansible -i ./multinode bootstrap-servers
# Pre-deployment environment check, may report docker version errors, can be ignored
kolla-ansible -i ./multinode prechecks
# Pull images, this step can be omitted, it will automatically pull by default
kolla-ansible -i ./multinode pull
# Execute actual deployment, pull images, run corresponding component containers
kolla-ansible -i ./multinode deploy
# Generate openrc file
kolla-ansible post-deploy

OpenStack Client

docker run -d --name client \  
  --restart always \  
  -v /etc/kolla/admin-openrc.sh:/admin-openrc.sh:ro \  
  -v /usr/share/kolla-ansible/init-runonce:/init-runonce:rw \  
  kolla-register.ssturing.com/kolla/centos-binary-openstack-base:train sleep infinity

Skyline

skyline.yaml template file:

https://docs.openstack.org/skyline-apiserver/latest/configuration/settings.html#configuration-settings

docker exec -it client bash
source /admin-openrc.sh
openstack service list
openstack user create --domain default --password-prompt skyline
User Password: # Custom password here
skyline
Repeat User Password: # Custom password here
skyline
openstack role add --project service --user skyline admin
# View database password
cat /etc/kolla/passwords.yml | grep ^database
docker exec -it  mariadb  bash  # Operate MariaDB
[(none)]> CREATE DATABASE IF NOT EXISTS skyline DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
MariaDB [(none)]> GRANT ALL PRIVILEGES ON skyline.* TO 'skyline'@'localhost' IDENTIFIED BY 'skyline';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON skyline.* TO 'skyline'@'%'  IDENTIFIED BY 'skyline';
mkdir -p /etc/skyline /var/log/skyline /var/lib/skyline /var/log/nginx /etc/skyline/policy
vim /etc/skyline/skyline.yaml and modify the corresponding lines...
database_url: mysql://root:[email protected]:3306/skyline
debug: true
log_dir: /var/log/skyline
prometheus_endpoint: http://192.168.1.100:9091
keystone_url: http://192.168.1.100:5000/v3/
system_project: service
system_user_password: 'skyline'...
If you see the following log information, it indicates that the bootstrap server initialization was successful
[root@controller1 ~]# docker run -d --name skyline_bootstrap -e KOLLA_BOOTSTRAP="" -v /etc/skyline/skyline.yaml:/etc/skyline/skyline.yaml -v /var/log:/var/log --net=host kolla-register.ssturing.com/99cloud/skyline:latest
706c39c1eb7194581d1093e7f413057ee59b547c681c0d138530b7657922c810
[root@controller1 ~]# docker logs skyline_bootstrap
+ echo '/usr/local/bin/gunicorn -c /etc/skyline/gunicorn.py skyline_apiserver.main:app'
+ mapfile -t CMD
+ tail /run_command
+ xargs -n 1
+ [[ -n 0 ]]
+ cd /opt/skyline_apiserver/
+ make db_sync
alembic -c skyline_apiserver/db/alembic/alembic.ini upgrade head
2025-05-28 03:52:51.542 | INFO     | alembic.runtime.migration:__init__:205 - Context impl MySQLImpl.
2025-05-28 03:52:51.542 | INFO     | alembic.runtime.migration:__init__:208 - Will assume non-transactional DDL.
2025-05-28 03:52:51.550 | INFO     | alembic.runtime.migration:run_migrations:619 - Running upgrade  -> 000, init
+ exit 0
docker rm -f skyline_bootstrap
docker run -d --name skyline --restart=always -v /etc/skyline/skyline.yaml:/etc/skyline/skyline.yaml -v /var/log:/var/log --net=host kolla-register.ssturing.com/99cloud/skyline:latest

Access Skyline

Address: http://192.168.1.100:9999

Technical Discussion: High Availability with Kolla-Ansible

Grafana

Address: http://192.168.1.100:3000

Username: admin

Password: cat /etc/kolla/passwords.yml | grep ^grafana_admin

Technical Discussion: High Availability with Kolla-AnsibleTechnical Discussion: High Availability with Kolla-Ansible

Visit https://grafana.com/grafana/dashboards to find corresponding dashboard templates, such as OpenStack Dashboard: 21085

For more information on OpenStack optimization deployment tools, please follow this public account for future updates.

Please follow the “Shensu Zhihuan” public account to learn more about cooperation dynamics and technical information.

Leave a Comment