Installing Object Storage (Swift) manually

With IBM® Cloud Manager with OpenStack version 4.3, you cannot install the OpenStack Object Storage (Swift) project by using a cookbook as part of the deployment process. You must install the OpenStack Object Storage (Swift) project manually.

Before you begin

Before you can manually install Swift, the following tasks must be complete:
  • IBM Cloud Manager with OpenStack is already installed.
  • A cloud environment is already deployed.
  • There is a separate node to install Swift.

About this task

Note: IBM Cloud Manager with OpenStack supports English only for the OpenStack Object Storage (Swift) project.
Use the following procedure to install swift manually.

Procedure

  1. Configure yum repositories.
    Before you install Swift on your Swift node, your Swift node must have access to yum repositories that contain Swift packages that IBM shipped. Use the following procedures to configure yum repositories for the Swift node:
    1. Log in to the deployment system as the root user. This is the system where IBM Cloud Manager with OpenStack was installed.
    2. Copy the following example topology to a file, your-topology-name.json. Change your-topology-name to the name for your topology. Here is an example topology.
      {
  "name":"CHANGEME”,
  "description":"CHANGEME”,
  "environment":"CHANGEME”,
  "secret_file":"CHANGEME”,
  "run_sequentially":false,
  "nodes": [
    {
      "fqdn":"CHANGEME”,
      "password":"CHANGEME”,
       "identity_file":"CHANGEME",
      "quit_on_error":true,
      "run_order_number":1,
      "runlist": [
        "role[ibm-os-base]"
      ]
}
  ]
}
    3. Customize the topology file. Change the following JSON attributes in your topology file, your-topology-name.json:
      • Name: Set to your topology name: your-topology-name.
      • Description: Set to the description for your topology.
      • Environment: Set to the environment for your topology: your-environment-name.
      • nodes.fqdn: Set to the fully qualified domain name of the Swift node. The deployment system must be able to ssh using the fully qualified domain name. You can also set to the public IP address, private IP address, or host name.
      • nodes.password or nodes.identity_file: For each node, set to the appropriate SSH root user authentication for the node system. Either a password or an SSH identity file can be used for authentication. Remove the unused attribute for each node.
      For secret_file customization, if needed, see Customizing passwords and secrets.
    4. Deploy the topology to configure yum repositories for the Swift node:
      knife os manage deploy topology your-topology-name.json
  2. Run the following two shell scripts for the Swift installation. The first script (Script 1) is for the Swift user, and must be run from the controller node. Script two (Script 2) is for the Swift node configuration, and must be run from the Swift node. Assign relevant values to the following variables: IP_ADDR, KEYSTONE_IP, and REGION. You can customize these values according to your environment needs.
    Note: The {{***}} needs to be replaced with your environment setting, like controller.ip, os_token, etc.

    For Script 1, the variable KEYSTONE_IP and IP_ADDR must be changed to an actual IP address.

    For Script 2, the variable KEYSTONE_IP and IP_ADDR, must be changed to an actual IP address.

    For other settings,
    • SERVICE_PROJECT_NAME: If your Keystone server does not have a “service” project (default setting), you must set this value according to your environment's specific configuration. It is the same project that is used in Nova, Glance, and so on.
    • SWIFT_PWD: Customize this value if you want to use other passwords. Be sure that the value is consistent across Script 1 and Script 2.
    • PORT: Enter the Swift proxy server port. If your Swift node is the same as your IBM Cloud Manager - Self Service node, then change it to a different port, since the IBM Cloud Manager - Self Service uses port 8080, by default. Be sure that the PORT setting is consistent across Script 1 and Script 2.
    
Script 1
#!/bin/sh
echo "Running $0"
set -x

function get_id {
    echo `"$@" | grep ' id ' | awk '{print $4}'`
}

KEYSTONE_IP={{controller.ip}}
# Get service token from keystone.conf, and decrypted the token
export OS_TOKEN=`grep "^[[:space:]]*admin_token" /etc/keystone/keystone.conf  |
awk -F'='  '{print $2}' | sed  's/^[[:space:]]*//' | xargs openstack-obfuscate  -u`

export OS_IDENTITY_API_VERSION=3
export OS_URL="http://$KEYSTONE_IP:35357/v3"
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_DOMAIN_ID=default

IP_ADDR={{swift node ip}}
SERVICE_PROJECT_NAME=service
SWIFT_PWD=swift
REGION=RegionOne
#Change this port to other available port if Self-Service Portal(SSP) used 8080 port.
PORT=8080
VERSION_SWIFT=v1
ADMIN_ROLE_ID=$(get_id openstack role show admin)

# Setting up tenants.
echo "Setting up tenants..."
# Attempt to get the SERVICE_PROJECT_ID.
SERVICE_PROJECT_ID=$(get_id openstack project show $SERVICE_PROJECT_NAME)
if [ -z $SERVICE_PROJECT_ID ]; then
    # Create the tenant if none existed.
    SERVICE_PROJECT_ID=$(get_id openstack project create --name $SERVICE_PROJECT_NAME --description 'Service Tenant')
fi
echo "SERVICE_PROJECT_ID='$SERVICE_PROJECT_ID'"
if [ -z $SERVICE_PROJECT_ID ]; then
    echo "Unable to get the SERVICE_PROJECT_ID from $SERVICE_PROJECT_NAME."
    echo "The script can not continue without a valid id."
    echo "Fix the problem with keystone and run the script again."
    exit 1
fi

# Add the swift service user to keystone
SWIFT_USER_ID=$(get_id openstack user create --domain $OS_USER_DOMAIN_ID --project $SERVICE_PROJECT_ID --password $SWIFT_PWD swift)
openstack role add --project $SERVICE_PROJECT_ID --user $SWIFT_USER_ID $ADMIN_ROLE_ID

# for tempest test case
openstack role create ResellerAdmin

# Add the swift endpoint to keystone
SWIFT_SERVICE_ID=$(get_id openstack service create --name=swift --description="Object Storage Service" object-store)

openstack endpoint create --region $REGION $SWIFT_SERVICE_ID public 'http://'$IP_ADDR':'$PORT'/'$VERSION_SWIFT'/AUTH_%(tenant_id)s'

openstack endpoint create --region $REGION $SWIFT_SERVICE_ID internal 'http://'$IP_ADDR':'$PORT'/'$VERSION_SWIFT'/AUTH_%(tenant_id)s'

openstack endpoint create --region $REGION $SWIFT_SERVICE_ID admin 'http://'$IP_ADDR':'$PORT'/'$VERSION_SWIFT'/AUTH_%(tenant_id)s'
    
Script 2: (Run from the Swift node), 
#!/bin/sh
echo "Running $0"
set -x

KEYSTONE_IP={{controller.ip}}
IP_ADDR={{swift node ip}}
SERVICE_TENANT_NAME=service
SWIFT_PWD=swift
#Change this port to other available port if Self-Service Portal(SSP) used 8080 port.
PORT=8080

ARCH="NA"
archString=$(/bin/uname -a)
if [[ $archString == *ppc64* ]]; then
    ARCH="ppc64"
elif [[ $archString == *x86_64* ]];then
    ARCH="x86_64"
elif [[ $archString == *s390x* ]];then
    ARCH="s390x"
fi

# Add the swift user to the system
groupadd swift
useradd -g swift swift

#
# Installing and Configuring the Object Service
#
# Set the OS_USERNAME and OS_PASSWORD with Swift values for the setup scripts.
export OS_PASSWORD=swift
export OS_USERNAME=swift
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=service
export OS_PROJECT_DOMAIN_ID=default
export OS_AUTH_URL="http://$KEYSTONE_IP:5000/v3"
export OS_IDENTITY_API_VERSION=3

# Install Openstack Swift storage packages
yum -y install openstack-swift-account openstack-swift-container openstack-swift-object
if [ $? -ne 0 ]; then
    echo "Could not install Swift storage packages."; exit 1;
fi
# Install Openstack Swift proxy packages
yum -y install openstack-swift-proxy
if [ $? -ne 0 ]; then
    echo "Could not install Swift proxy packages."; exit 1;
fi

# Set some random string of text as the suffix value.
RANDOM_STRING=$(cat /dev/urandom | head -n 15 | md5sum | head -c 15)
openstack-config --set /etc/swift/swift.conf swift-hash swift_hash_path_suffix $RANDOM_STRING

# Install xfs packages
yum -y install xfsdump xfsprogs-devel xfsprogs
XFS=1
if [ $? -ne 0 ]; then
    echo "Could not install XFS packages.";
    XFS=0
fi

# Create Swift backing disk
SWIFT_DATA_DIR=/srv/node
SWIFT_DISK_IMAGE=${SWIFT_DATA_DIR}/drives/images/swift.img
SWIFT_MOUNT_POINT=${SWIFT_DATA_DIR}/sdb1
mkdir -p ${SWIFT_MOUNT_POINT}
mkdir -p ${SWIFT_DATA_DIR}/drives
SWIFT_DRIVES_DIR=${SWIFT_DATA_DIR}/drives

# Create a loopback disk and format it to XFS
if [[ -e ${SWIFT_DISK_IMAGE} ]]; then
    if egrep -q ${SWIFT_MOUNT_POINT} /proc/mounts; then
        umount ${SWIFT_MOUNT_POINT}
        rm -f ${SWIFT_DISK_IMAGE}
    fi
fi

function write_to {
    # RTC 178288 : Checking the duplicated entries
    # Pass "check" as the 3rd argument to only write if the value isn't there already.
    if [ -f $2 ] && [[ $# -eq 3 ]] && [[ "$3" == 'check' ]]; then
        grep -q $1 $2
        if [ $? -eq 0 ]; then
            # do nothing
            return
        fi
    fi
    echo $1 >> $2
}

# create the file for the loopback device, modify size  specified in the truncate command to 
make a larger or smaller partition as needed.  In our example, we set it 1G.
mkdir -p ${SWIFT_DATA_DIR}/drives/images
touch ${SWIFT_DISK_IMAGE}
truncate -s 1G ${SWIFT_DISK_IMAGE}


# Mount the disk with mount options to make it as efficient as possible
mkdir -p ${SWIFT_MOUNT_POINT}
if [ $XFS -ne 0 ]; then
    # Make a fresh XFS filesystem
    mkfs.xfs -f -i size=1024 ${SWIFT_DISK_IMAGE}
    write_to "${SWIFT_DISK_IMAGE} ${SWIFT_MOUNT_POINT} xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0" /etc/fstab
    mount ${SWIFT_MOUNT_POINT}
else
    # Make a fresh ext4 filesystem
    mkfs.ext4 -F -I 1024 ${SWIFT_DISK_IMAGE}
    mount -t ext4 -o loop ${SWIFT_DISK_IMAGE} ${SWIFT_MOUNT_POINT}
fi
#Change swift owner and groups
chown -R swift:swift ${SWIFT_DATA_DIR}

#If enabled selinux, it may need to run this to prevent permission issue in selinux 
restorecon -R  ${SWIFT_DATA_DIR}

# Configure rsync service
RSYNC_CONF_FILE=/etc/rsyncd.conf
if [ -f $RSYNC_CONF_FILE ]; then
    rm -f $RSYNC_CONF_FILE
fi


write_to "uid = swift" $RSYNC_CONF_FILE
write_to "gid = swift" $RSYNC_CONF_FILE
write_to "log file = /var/log/rsyncd.log" $RSYNC_CONF_FILE
write_to "pid file = /var/run/rsyncd.pid" $RSYNC_CONF_FILE
write_to "address = $IP_ADDR" $RSYNC_CONF_FILE
write_to "[account]" $RSYNC_CONF_FILE
write_to "max connections = 2" $RSYNC_CONF_FILE
write_to "path = ${SWIFT_DATA_DIR}/drives/" $RSYNC_CONF_FILE
write_to "read only = false" $RSYNC_CONF_FILE
write_to "lock file = /var/lock/account.lock" $RSYNC_CONF_FILE
write_to "[container]" $RSYNC_CONF_FILE
write_to "max connections = 2" $RSYNC_CONF_FILE
write_to "path = ${SWIFT_DATA_DIR}/drives/" $RSYNC_CONF_FILE
write_to "read only = false" $RSYNC_CONF_FILE
write_to "lock file = /var/lock/container.lock" $RSYNC_CONF_FILE
write_to "[object]" $RSYNC_CONF_FILE
write_to "max connections = 2" $RSYNC_CONF_FILE
write_to "path = ${SWIFT_DATA_DIR}/drives/" $RSYNC_CONF_FILE
write_to "read only = false" $RSYNC_CONF_FILE
write_to "lock file = /var/lock/object.lock" $RSYNC_CONF_FILE

DEFAULT_RSYNC=/etc/default/rsync

write_to "RSYNC_ENABLE=true" $DEFAULT_RSYNC "check"

/usr/bin/rsync --daemon
if [ $? -ne 0 ]; then
    echo "The rsync service failed to start."; exit 1;
fi

# Configure memcached service
yum -y install memcached
if [ $? -ne 0 ]; then
    echo "Could not install memcached!"; exit 1;
fi
MEMCACHED_CONF_FILE=/etc/memcached.conf

write_to "-l $IP_ADDR" $MEMCACHED_CONF_FILE "check"

service memcached restart
if [ $? -ne 0 ]; then
    echo "Could not start memcached!"; exit 1;
fi
#  Install the auth middleware for swift and utils for easy configuration
yum -y install openstack-utils
yum -y install keystonemiddleware

SWIFT_PROXY_CONFIG_FILE=/etc/swift/proxy-server.conf
SWIFT_ACCOUNT_CONFIG_FILE=/etc/swift/account-server.conf
SWIFT_CONTAINER_CONFIG_FILE=/etc/swift/container-server.conf
SWIFT_OBJECT_CONFIG_FILE=/etc/swift/object-server.conf

 Configure Swift proxy server
openstack-config --set $SWIFT_PROXY_CONFIG_FILE DEFAULT bind_port $PORT
openstack-config --set $SWIFT_PROXY_CONFIG_FILE DEFAULT bind_ip $IP_ADDR
openstack-config --set $SWIFT_PROXY_CONFIG_FILE DEFAULT swift_dir /etc/swift
openstack-config --set $SWIFT_PROXY_CONFIG_FILE DEFAULT workers 1
openstack-config --set $SWIFT_PROXY_CONFIG_FILE app:proxy-server account_autocreate true
openstack-config --set $SWIFT_PROXY_CONFIG_FILE app:proxy-server conn_timeout 20
openstack-config --set $SWIFT_PROXY_CONFIG_FILE app:proxy-server node_timeout 120
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:keystone operator_roles "_member_, admin"
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken signing_dir /var/cache/swift
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken auth_host $KEYSTONE_IP
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken admin_tenant_name $SERVICE_TENANT_NAME
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken admin_user swift
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken admin_password $SWIFT_PWD
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken auth_uri http://$KEYSTONE_IP:5000
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken auth_version v3.0
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken delay_auth_decision true
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:tempurl use egg:swift#tempurl
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:crossdomain use egg:swift#crossdomain
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:ratelimit use egg:swift#ratelimit
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:bulk use egg:swift#bulk
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:container_sync use egg:swift#container_sync

openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:tempauth use egg:swift#tempauth
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:tempauth reseller_prefix TEMPAUTH
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:staticweb use egg:swift#staticweb
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:formpost use egg:swift#formpost

openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:container-quotas use egg:swift#container_quotas
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:account-quotas use egg:swift#account_quotas
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:slo use egg:swift#slo
openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:dlo use egg:swift#dlo

openstack-config --set $SWIFT_PROXY_CONFIG_FILE filter:authtoken auth_uri http://$KEYSTONE_IP:5000
openstack-config --set $SWIFT_PROXY_CONFIG_FILE pipeline:main pipeline "healthcheck cache container_sync 
bulk tempurl crossdomain authtoken keystone tempauth formpost staticweb container-quotas account-quotas proxy-server"
# Create the account, container and object rings
cd /etc/swift
swift-ring-builder account.builder create 18 3 1
if [ $? -ne 0 ]; then
    echo "Failed to create account ring."; exit 1;
fi
swift-ring-builder container.builder create 18 3 1
if [ $? -ne 0 ]; then
    echo "Failed to create container ring."; exit 1;
fi
swift-ring-builder object.builder create 18 3 1
if [ $? -ne 0 ]; then
    echo "Failed to create object ring."; exit 1;
fi

# For every storage device, add entries to each ring
swift-ring-builder account.builder add z1-$IP_ADDR:6002R$IP_ADDR:6005/sdb1 100
if [ $? -ne 0 ]; then
    echo "Failed to add entry to account ring."; exit 1;
fi
swift-ring-builder container.builder add z1-$IP_ADDR:6001R$IP_ADDR:6004/sdb1 100
if [ $? -ne 0 ]; then
    echo "Failed to add entry to container ring."; exit 1;
fi
swift-ring-builder object.builder add z1-$IP_ADDR:6000R$IP_ADDR:6003/sdb1 100
if [ $? -ne 0 ]; then
    echo "Failed to add entry to object ring."; exit 1;
fi

# Verify the rings
swift-ring-builder account.builder
swift-ring-builder container.builder
swift-ring-builder object.builder

# Rebalance the rings
swift-ring-builder account.builder rebalance
if [ $? -ne 0 ]; then
    echo "Failed to rebalance account ring."; exit 1;
fi
swift-ring-builder container.builder rebalance
if [ $? -ne 0 ]; then
    echo "Failed to rebalance container ring."; exit 1;
fi
swift-ring-builder object.builder rebalance
if [ $? -ne 0 ]; then
    echo "Failed to rebalance object ring."; exit 1;
fi
#Configure Swift account server
openstack-config --set $SWIFT_ACCOUNT_CONFIG_FILE DEFAULT bind_ip $IP_ADDR
openstack-config --set $SWIFT_ACCOUNT_CONFIG_FILE DEFAULT mount_check false
openstack-config --set $SWIFT_ACCOUNT_CONFIG_FILE DEFAULT disable_fallocate true
openstack-config --set $SWIFT_ACCOUNT_CONFIG_FILE DEFAULT workers 1
openstack-config --set $SWIFT_ACCOUNT_CONFIG_FILE DEFAULT devices $SWIFT_DRIVES_DIR

#Configure Swift container server
openstack-config --set $SWIFT_CONTAINER_CONFIG_FILE DEFAULT bind_ip $IP_ADDR
openstack-config --set $SWIFT_CONTAINER_CONFIG_FILE DEFAULT mount_check false
openstack-config --set $SWIFT_CONTAINER_CONFIG_FILE DEFAULT disable_fallocate true
openstack-config --set $SWIFT_CONTAINER_CONFIG_FILE DEFAULT workers 1
openstack-config --set $SWIFT_CONTAINER_CONFIG_FILE DEFAULT devices $SWIFT_DRIVES_DIR
openstack-config --set $SWIFT_CONTAINER_CONFIG_FILE DEFAULT allow_versions true

#Configure Swift object server
openstack-config --set $SWIFT_OBJECT_CONFIG_FILE DEFAULT bind_ip $IP_ADDR
openstack-config --set $SWIFT_OBJECT_CONFIG_FILE DEFAULT mount_check false
openstack-config --set $SWIFT_OBJECT_CONFIG_FILE DEFAULT disable_fallocate true
openstack-config --set $SWIFT_OBJECT_CONFIG_FILE DEFAULT workers 1
openstack-config --set $SWIFT_OBJECT_CONFIG_FILE DEFAULT devices $SWIFT_DRIVES_DIR
#config Swift container sync realms
SWIFT_CSYNC_CONFIG_FILE=/etc/swift/container-sync-realms.conf
touch SWIFT_CSYNC_CONFIG_FILE
openstack-config --set $SWIFT_CSYNC_CONFIG_FILE realm1 key realm1key
openstack-config --set $SWIFT_CSYNC_CONFIG_FILE realm1 realm1 cluster_name1 "http://$IP_ADDR:$PORT/v1/"
chown -R swift:swift /etc/swift

#Start all Swift services
systemctl enable memcached.service rsyncd.service
SWIFT_ACCOUNT_SERVICES="account account-auditor account-reaper account-replicator"
for svc in $SWIFT_ACCOUNT_SERVICES; do
    systemctl enable openstack-swift-$svc;
    systemctl start openstack-swift-$svc;
done

SWIFT_CONTAINER_SERVICES="container container-replicator container-updater container-auditor"
for svc in $SWIFT_CONTAINER_SERVICES; do
    systemctl enable openstack-swift-$svc;
    systemctl start openstack-swift-$svc;
done

SWIFT_OBJECT_SERVICES="object object-replicator object-updater object-auditor"
for svc in $SWIFT_OBJECT_SERVICES; do
    systemctl enable openstack-swift-$svc;
    systemctl start openstack-swift-$svc;
done

systemctl enable openstack-swift-proxy
systemctl start openstack-swift-proxy

#If enabled selinux, it may need to run this to prevent permission issue in selinux 
restorecon -R /var/cache/swift 

echo "Swift script complete"
  3. After running the shell script to install Swift successfully, you can verify Swift operation. If you run Swift operations from the controller node, you must be sure iptables is configured in the Swift node to allow that access, or else, the controller node cannot access the Swift node services.
    For example, run the following commands, from the Swift node, replacing <swift proxy port> and <control node hostname> according to the settings in your environment.
    #iptables -I  INPUT -p tcp -m tcp --dport <swift proxy port> -s <control node hostname> -j ACCEPT 
# service iptables save 
# systemctl restart iptables.service
  4. Open a new terminal and source openrc file.
    Run the following Swift commands from the controller node:
    1. Show the Swift service.
      $ swift stat
        Account: AUTH_3b1631f3adfb4aa691d1af0bf3d00867
     Containers: 0
        Objects: 0
          Bytes: 0
X-Put-Timestamp: 1433837690.54122
    X-Timestamp: 1433837690.54122
     X-Trans-Id: tx430af05c0fde46b79167f-005576a07a
   Content-Type: text/plain; charset=utf-8
    2. Upload a test file and replace FILE with your local file name.
      $ swift upload demo-container1 FILE
    3. List containers.
      $ swift list
demo-container1
    4. Download a test file and replace FILE with name used in step 4b above.
      $swift download  demo-container1 FILE 
FILE  [auth 0.236s, headers 0.336s, total 0.336s, 0.014 MB/s]
      Step 4 assumes you use openrc for the v2.0 keystone credential. If you use v3.0 keystone credentials in openrc, you must issue each Swift command with “-V 3”. For example,
      $ swift -V 3 list
demo-container1