Topic
  • 5 replies
  • Latest Post - ‏2013-01-25T06:11:59Z by Rphilo
AbhilashJoseph
AbhilashJoseph
36 Posts

Pinned topic Staging Store Disk Quota

‏2013-01-22T23:24:40Z | idr-cdc
 Hi 
 
We had an incident of staging_store_disk_quota_gb of 10 GB being full and and CDC running dead slow and hence we had to reconfigure to 20 GB .Ours is s source database with around  100 GB of logs per day. I have seen again that staging store has gone 47% Full even though CDC is not running without hardly any latency ( 5 minutes and archive logs only ) . I have enabled continuous capture for the instance, is it contributing to  staging store getting filled faster? 
Is there a need to regularly stop and start mirroring to avoid staging store getting full ?  We do not have any more disk to be allocated as we were told that 10 GB was too good for 200 GB log  by IBM Consultant.
 When does the staging store start releasing  disk space ?
 
 
 
  • elainep
    elainep
    22 Posts

    Re: Staging Store Disk Quota

    ‏2013-01-23T21:55:11Z  
     Hi,
    Is there a particular reason you chose to enable  continuous capture for the instance?  If continuous capture is enabled and you have stopped your subscriptions CDC will continue to read and 'stage' the inscope changes into transaction queues that hit disk.  If you disable continuous capture CDC will not do any reading when all subscriptions are not running.  When you restart replication CDC will read the changes from the archive logs (given that they are still available).  CDC releases the changes from the stagingstore once the operations for all applicable subscriptions has been sent to the target.  If you see that it is not being released it could be because:
    - transaction has not been committed
    - there may be an idle subscription that still requires the operation
     
    Usually you would allocate sufficient disk space to keep the maximum storage of concurrent transactions before they are committed and then sent to the target.
  • AbhilashJoseph
    AbhilashJoseph
    36 Posts

    Re: Staging Store Disk Quota

    ‏2013-01-23T23:08:06Z  
    • elainep
    • ‏2013-01-23T21:55:11Z
     Hi,
    Is there a particular reason you chose to enable  continuous capture for the instance?  If continuous capture is enabled and you have stopped your subscriptions CDC will continue to read and 'stage' the inscope changes into transaction queues that hit disk.  If you disable continuous capture CDC will not do any reading when all subscriptions are not running.  When you restart replication CDC will read the changes from the archive logs (given that they are still available).  CDC releases the changes from the stagingstore once the operations for all applicable subscriptions has been sent to the target.  If you see that it is not being released it could be because:
    - transaction has not been committed
    - there may be an idle subscription that still requires the operation
     
    Usually you would allocate sufficient disk space to keep the maximum storage of concurrent transactions before they are committed and then sent to the target.
     Hi Elaine,
     
    I have enabled staging store continuous capture as our log retention is only 24 hours and doesnt want to loose any logs even if our target is down.We have only one subscription from the instance and was always active and writing to target and as per staging store scrapped point and committed point are current which means all changes have been applier to target .See example below we restarted subscription on 21st evening 
     

     

    -bash-3.2$ ./dmgetstagingstorestatus  -I EIPPROMU

    The Staging Store is enabled.

    Continuous Capture is enabled.

    Capture is running.

    1 out of 1 subscriptions are running.

    Estimated disk space reserved for in-memory data is 93,323,176 bytes.

    Actual disk space usage is 11,151,790,107 bytes.

    The Staging Store is 52% full.

    Capture latency is 86 sec.

    Staging store oldest operation bookmark: Journal name JOURNAL Journal bookmark 000308;7420128131498;7420141537495.1.1.149680.224.1;7420141537495.1.1.149680.224. 1.0

     

    Staging store newest operation bookmark: Journal name JOURNAL Journal bookmark 000308;7420131288121;7420265567992.1.1.2025084.252.1;7420265567992.1.1.2025084.252.1.0

     

    Which is

     

    Oldest operation bookmark  21/01/2013 9:30:46.000000000 AM  , 21/01/2013 6:39:39.000000000 PM, 21/01/2013 6:39:39.000000000 PM

     

    newest operation bookmark  21/01/2013 11:25:07.000000000 AM, 24/01/2013 9:27:22.000000000 AM, 24/01/2013 9:27:22.000000000 AM

     

    There is a pending transaction at 21/01/2013 11:25:07.000000000 AM which explains why the restart point is still of 21st but why should the data store be 53% in this case ? 
  • Rphilo
    Rphilo
    502 Posts

    Re: Staging Store Disk Quota

    ‏2013-01-24T21:50:42Z  
     Hi Elaine,
     
    I have enabled staging store continuous capture as our log retention is only 24 hours and doesnt want to loose any logs even if our target is down.We have only one subscription from the instance and was always active and writing to target and as per staging store scrapped point and committed point are current which means all changes have been applier to target .See example below we restarted subscription on 21st evening 
     

     

    -bash-3.2$ ./dmgetstagingstorestatus  -I EIPPROMU

    The Staging Store is enabled.

    Continuous Capture is enabled.

    Capture is running.

    1 out of 1 subscriptions are running.

    Estimated disk space reserved for in-memory data is 93,323,176 bytes.

    Actual disk space usage is 11,151,790,107 bytes.

    The Staging Store is 52% full.

    Capture latency is 86 sec.

    Staging store oldest operation bookmark: Journal name JOURNAL Journal bookmark 000308;7420128131498;7420141537495.1.1.149680.224.1;7420141537495.1.1.149680.224. 1.0

     

    Staging store newest operation bookmark: Journal name JOURNAL Journal bookmark 000308;7420131288121;7420265567992.1.1.2025084.252.1;7420265567992.1.1.2025084.252.1.0

     

    Which is

     

    Oldest operation bookmark  21/01/2013 9:30:46.000000000 AM  , 21/01/2013 6:39:39.000000000 PM, 21/01/2013 6:39:39.000000000 PM

     

    newest operation bookmark  21/01/2013 11:25:07.000000000 AM, 24/01/2013 9:27:22.000000000 AM, 24/01/2013 9:27:22.000000000 AM

     

    There is a pending transaction at 21/01/2013 11:25:07.000000000 AM which explains why the restart point is still of 21st but why should the data store be 53% in this case ? 
     Abhilash
     
    Given that there are no pending transactions and no idle subscriptions I cannot explain why the stagingstore is full and I would recommend that you open up a PMR
     
    One thing you should be aware of  - enabling continuous capture is not a guaranteed solution where the retention period for archive logs is not long enough for all cases in CDC. The staging store is essentially a memory structure and if replication ends abnormally for any reason, then the staging store would be corrupted and you would not be able to clear it. The only soluition that can be recommended is to have the archive logs manageed by CDC with the dmshowlogdependcy command on the source. There are cases where the archive logs are managed by RMAN and this management cannot be changed; in this case a possible solution is to duplex the archive logs and have the second archive log location managed by CDC, leaving the existing management processes for the first archive log location.
     
    Regards
     
    Robert
  • AbhilashJoseph
    AbhilashJoseph
    36 Posts

    Re: Staging Store Disk Quota

    ‏2013-01-25T00:08:04Z  
    • Rphilo
    • ‏2013-01-24T21:50:42Z
     Abhilash
     
    Given that there are no pending transactions and no idle subscriptions I cannot explain why the stagingstore is full and I would recommend that you open up a PMR
     
    One thing you should be aware of  - enabling continuous capture is not a guaranteed solution where the retention period for archive logs is not long enough for all cases in CDC. The staging store is essentially a memory structure and if replication ends abnormally for any reason, then the staging store would be corrupted and you would not be able to clear it. The only soluition that can be recommended is to have the archive logs manageed by CDC with the dmshowlogdependcy command on the source. There are cases where the archive logs are managed by RMAN and this management cannot be changed; in this case a possible solution is to duplex the archive logs and have the second archive log location managed by CDC, leaving the existing management processes for the first archive log location.
     
    Regards
     
    Robert
     Ho Robert,
     
    Do you think continuous capture can add to  staging store getting full.
    The interesting thing is when I stopped subscription and restarted the instance, the staging store got emptied ie I didnt have to  clear the staging store or set the log position.
     
    Regards 
    Abhilash  
  • Rphilo
    Rphilo
    502 Posts

    Re: Staging Store Disk Quota

    ‏2013-01-25T06:11:59Z  
     Ho Robert,
     
    Do you think continuous capture can add to  staging store getting full.
    The interesting thing is when I stopped subscription and restarted the instance, the staging store got emptied ie I didnt have to  clear the staging store or set the log position.
     
    Regards 
    Abhilash  
     Abhilash 
     
    I am not sure why the staging store does not get emptied until you stop the instance, but continuous capture is a factor in this.
     
    You do not have to set the log position under normal circumstances when restarting replication, only if you need to specifically set the log position for a subscription, for example because you have split a subscription into multiple subscriptions.
     
    Regards
     
    Robert