Replacing the OSDs with pre-created LVM

After purging the OSD with the ceph-volume lvm zap command, if the directory is not present, then you can replace the OSDs with the OSD service specification file with the pre-created LVM.

Prerequisites

A running IBM Storage Ceph cluster.
Failed OSD

Procedure

Log into the cephadm shell:

Example
```
 [root@host01 ~]# cephadm shell
```
Set the OSD service to an unmanaged state to prevent cephadm from automatically redeploying OSDs during removal.
Note: If you skip this step, cephadm may redeploy OSDs automatically, causing conflicts with the removal operation.

Example
```
service_type: osd
service_id: <service_id>
unmanaged: true
```
Apply the specification:

Example
```
ceph orch apply -i <osds.yaml>
```
Run the orch ls ceph command to confirm that the OSD service is set to the unmanaged state.

Remove the OSD:

Syntax

 ceph orch osd rm OSD_ID [--replace]

Example

 [ceph: root@host01 /]# ceph orch osd rm 8 --replace
 Scheduled OSD(s) for removal

Verify the OSD is destroyed:

Example

   [ceph: root@host01 /]# ceph osd tree

   ID   CLASS  WEIGHT   TYPE NAME        STATUS     REWEIGHT  PRI-AFF
   -1         0.32297  root default
   -9         0.05177      host host10
    3    hdd  0.01520          osd.3           up   1.00000  1.00000
   13    hdd  0.02489          osd.13          up   1.00000  1.00000
   17    hdd  0.01169          osd.17          up   1.00000  1.00000
   -13         0.05177      host host11
    2    hdd  0.01520          osd.2           up   1.00000  1.00000
   15    hdd  0.02489          osd.15          up   1.00000  1.00000
   19    hdd  0.01169          osd.19          up   1.00000  1.00000
   -7         0.05835      host host12
   20    hdd  0.01459          osd.20          up   1.00000  1.00000
   21    hdd  0.01459          osd.21          up   1.00000  1.00000
   22    hdd  0.01459          osd.22          up   1.00000  1.00000
   23    hdd  0.01459          osd.23          up   1.00000  1.00000
   -5         0.03827      host host04
    1    hdd  0.01169          osd.1           up   1.00000  1.00000
    6    hdd  0.01129          osd.6           up   1.00000  1.00000
    7    hdd  0.00749          osd.7           up   1.00000  1.00000
    9    hdd  0.00780          osd.9           up   1.00000  1.00000
   -3         0.03816      host host05
    0    hdd  0.01169          osd.0           up   1.00000  1.00000
    8    hdd  0.01129          osd.8    destroyed         0  1.00000               
   12    hdd  0.00749          osd.12          up   1.00000  1.00000
   16    hdd  0.00769          osd.16          up   1.00000  1.00000
   -15         0.04237      host host06
    5    hdd  0.01239          osd.5           up   1.00000  1.00000
   10    hdd  0.01540          osd.10          up   1.00000  1.00000
   11    hdd  0.01459          osd.11          up   1.00000  1.00000
   -11         0.04227      host host07
    4    hdd  0.01239          osd.4           up   1.00000  1.00000
   14    hdd  0.01529          osd.14          up   1.00000  1.00000
   18    hdd  0.01459          osd.18          up   1.00000  1.00000

Zap and remove the OSD using the ceph-volume command:

Syntax

 ceph-volume lvm zap --osd-id OSD_ID

Example

 [ceph: root@host01 /]# ceph-volume lvm zap --osd-id 8

 Zapping: /dev/vg1/data-lv2
 Closing encrypted path /dev/mapper/l4D6ql-Prji-IzH4-dfhF-xzuf-5ETl-jNRcXC
 Running command: /usr/sbin/cryptsetup remove /dev/mapper/l4D6ql-Prji-IzH4-dfhF-xzuf-5ETl-jNRcXC
 Running command: /usr/bin/dd if=/dev/zero of=/dev/vg1/data-lv2 bs=1M count=10 conv=fsync
 stderr: 10+0 records in
 10+0 records out
 stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.034742 s, 302 MB/s
 Zapping successful for OSD: 8

Check the OSD topology:

Example

 [ceph: root@host01 /]# ceph-volume lvm list

Recreate the OSD with a specification file corresponding to that specific OSD topology:

Example

 [ceph: root@host01 /]# cat osd.yml
 service_type: osd
 service_id: osd_service
 placement:
   hosts:
   - host03
 data_devices:
   paths:
   - /dev/vg1/data-lv2
 db_devices:
   paths:
   - /dev/vg1/db-lv1

Apply the updated specification file:

Example
```
ceph orch apply -i osd.yml
```
Set the OSD service specification back to managed state. After you replace the disk, set the OSD service back to a managed state so that cephadm can resume orchestration.
1. Update the OSD service specification.
```
unmanaged: false
```
2. Apply the updated specification. This step allows cephadm to redeploy the OSD on the new device while reusing the existing OSD ID.
```
ceph orch apply -i <osds.yaml>
```

Verify the OSD is back:

Example

 [ceph: root@host01 /]# ceph -s
 [ceph: root@host01 /]# ceph osd tree