Question & Answer
Question
Use this document to collect PEX trace data for debugging high CPU usage related to cache line usage.
Answer
This document applies only to IBM i on Power7 (Note: IBM i 7.2 requires PTFs SI53586 and SI52743 be applied.)
Cache line traces are a specific type of PEX trace. These traces can be used to help isolate which jobs, programs, and procedures are changing cache lines and contributing to CPU consumption. The trace is obtained using Native OS/400 commands.
Optional Traces
Collect these additional traces only if requested by IBM Service.
Cache line traces are a specific type of PEX trace. These traces can be used to help isolate which jobs, programs, and procedures are changing cache lines and contributing to CPU consumption. The trace is obtained using Native OS/400 commands.
| 1. | Confirm that the partition is running on Power7 hardware. The PEX definitions are specific to the hardware: o DSPHDWRSC *PRC and press Enter. o Confirm that the listed Type-model, 9119-FHB in the example below, is listed here: http://www-01.ibm.com/support/knowledgecenter/POWER7/p7hdx/POWER7welcome.htm  |
| 2. | Confirm that all required PEX PTFs are applied: http://public.dhe.ibm.com/services/us/igsc/idoctor/html/ptfs.html |
| 3. | Add the PEX definition: ADDPEXDFN DFN(L3MISSPRF) LEVEL(*SERVICE) TYPE(*TRACE) JOB((*ALL *ALL)) TASK(*ALL) MAXSTG(102400000) PMC(232 2 *TASK *NONE *NONE *NONE 0000000000DE3F01) ITVSPEC(*COUNT) ITVCOUNT(10000) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT4)) TEXT('L3 Miss Format4 - Power7') |
| 4. | Collect the PEX data: a. Issue the CRTLIB PEXDTA command, and press Enter. b. Start the PEX trace: SBMJOB CMD(STRPEX SSNID(L3MISSPRF) DFN(L3MISSPRF)) JOBQ(QSYS/QSYSNOMAX) and press Enter. c. Run the workload. Gather for at least 10 minutes. This trace does not collect many events and can be run for as long as an hour. d. Stop the PEX trace: SBMJOB CMD(ENDPEX SSNID(L3MISSPRF) DTALIB(PEXDTA)) JOBQ(QSYS/QSYSNOMAX) and press Enter. e. The submitted jobs must complete before moving to the next step. Note: If requested by IBM service, collect additional Optional Traces as described below. |
| 5. | Save the data and send it to IBM: a. CRTSAVF QGPL/PEXDTA and press Enter. b. SBMJOB CMD(SAVLIB LIB(PEXDTA) DEV(*SAVF) SAVF(QGPL/PEXDATA)) JOB(SAVLIB) JOBQ(QSYS/QSYSNOMAX) c. FTP the save file named PEXDTA in library QGPL to IBM: http://www-01.ibm.com/support/docview.wss?uid=nas8N1010370 |
| 6. | Clean-up: o. DLTLIB PEXDTA and press Enter. |
Optional Traces
Collect these additional traces only if requested by IBM Service.
| 1. | DL2L3MOD4 o ADDPEXDFN DFN(DL2L3MOD4) LEVEL(*SERVICE) TYPE(*TRACE) JOB((*ALL *ALL)) TASK(*ALL) MAXSTG(102400000) PMC(237 3 *TASK *NONE *NONE *NONE 0000000000DE3F01) ITVSPEC(*COUNT) ITVCOUNT(10000) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT4)) TEXT('Distant L2 or L3 modified Format 4 - Power7') o STRPEX SSNID(MCLIRUN2) DFN(DL2L3MOD4) and press Enter. o Wait at least 5 minutes and then submit: SBMJOB CMD(ENDPEX SSNID(MCLIRUN2) DTAOPT(*MGTCOL) MGTCOL(PEXDTA/*SSNID)) JOB(ENDPEX) JOBQ(QSYSNOMAX) and press Enter. o This step must complete before moving to the next step. |
| 2. | DL2L3SHR4 o ADDPEXDFN DFN(DL2L3SHR4) LEVEL(*SERVICE) TYPE(*TRACE) JOB((*ALL *ALL)) TASK(*ALL) MAXSTG(102400000) PMC(233 3 *TASK *NONE *NONE *NONE 0000000000DE3F01) ITVSPEC(*COUNT) ITVCOUNT(10000) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT4)) TEXT('Distant L2 or L3 shared Format 4 - Power7') o STRPEX SSNID(MCLIRUN3) DFN(DL2L3SHR4) and press Enter. o Wait at least 5 minutes and submit: SBMJOB CMD(ENDPEX SSNID(MCLIRUN3) DTAOPT(*MGTCOL) MGTCOL(PEXDTA/*SSNID)) JOB(ENDPEX) JOBQ(QSYSNOMAX) and press Enter. o This step must complete before moving to the next step. |
| 3. | RL2L3MOD4 o ADDPEXDFN DFN(RL2L3MOD4) LEVEL(*SERVICE) TYPE(*TRACE) JOB((*ALL *ALL)) TASK(*ALL) MAXSTG(102400000) PMC(232 1 *TASK *NONE *NONE *NONE 0000000000DE3F01) ITVSPEC(*COUNT) ITVCOUNT(10000) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT4)) TEXT('Remote L2 or L3 modified Format 4 - Power7') o STRPEX SSNID(MCLIRUN4) DFN(RL2L3MOD4) and press Enter. o Wait at least 5 minutes and submit: SBMJOB CMD(ENDPEX SSNID(MCLIRUN4) DTAOPT(*MGTCOL) MGTCOL(PEXDTA/*SSNID)) JOB(ENDPEX) JOBQ(QSYSNOMAX) and press Enter. o This step must complete before moving to the next step. |
| 4. | RL2L3SHR4 o ADDPEXDFN DFN(RL2L3SHR4) LEVEL(*SERVICE) TYPE(*TRACE) JOB((*ALL *ALL)) TASK(*ALL) MAXSTG(102400000) PMC(234 1 *TASK *NONE *NONE *NONE 0000000000DE3F01) ITVSPEC(*COUNT) ITVCOUNT(10000) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT4)) TEXT('Remote L2 or L3 shared Format 4 - Power7') o STRPEX SSNID(MCLIRUN5) DFN(RL2L3SHR4) and press Enter. o Wait at least 5 minutes and submit: SBMJOB CMD(ENDPEX SSNID(MCLIRUN5) DTAOPT(*MGTCOL) MGTCOL(PEXDTA/*SSNID)) JOB(ENDPEX) JOBQ(QSYSNOMAX) and press Enter. o This step must complete before moving to the next step. |
| 5. | STRINTVN4 o ADDPEXDFN DFN(STRINTVN4) LEVEL(*SERVICE) TYPE(*TRACE) JOB((*ALL *ALL)) TASK(*ALL) MAXSTG(102400000) PMC(242 3 *TASK *NONE *NONE *NONE 0000000000DE3F01) ITVSPEC(*COUNT) ITVCOUNT(10000) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT4)) TEXT('Marked Store complete with intervention Format 4 - Power7') o STRPEX SSNID(MCLIRUN6) DFN(STRINTVN4) and press Enter. o Wait at least 5 minutes and submit: SBMJOB CMD(ENDPEX SSNID(MCLIRUN6) DTAOPT(*MGTCOL) MGTCOL(PEXDTA/*SSNID)) JOB(ENDPEX) JOBQ(QSYSNOMAX) and press Enter. o This step must complete before moving to the next step. |
| 6. | TPROFINST o ADDPEXDFN DFN(TPROFINST) LEVEL(*SERVICE) TYPE(*TRACE) JOB((*ALL *ALL)) TASK(*ALL) MAXSTG(10000000) ITVSPEC(*COUNT) ITVCOUNT(10000000) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT2)) o STRPEX SSNID(MCLIRUN7) DFN(TPROFINST) and press Enter. o Wait at least 5 minutes and submit: SBMJOB CMD(ENDPEX SSNID(MCLIRUN7) DTAOPT(*MGTCOL) MGTCOL(PEXDTA/*SSNID)) JOB(ENDPEX) JOBQ(QSYSNOMAX) and press Enter. o This step must complete before moving to the next step. |
| 7. | L3MISSUNM o ADDPEXDFN DFN(L3MISSUNM) LEVEL(*SERVICE) TYPE(*TRACE) JOB((*ALL *ALL)) TASK(*ALL) MAXSTG(102400000) PMC(97 2 *TASK *NONE *NONE *NONE) ITVSPEC(*COUNT) ITVCOUNT(10000) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT4)) TEXT('L3 Miss Unmarked Format4 - Power7') o STRPEX SSNID(MCLIRUN8) DFN(L3MISSUNM) and press Enter. o Wait at least 5 minutes and submit: SBMJOB CMD(ENDPEX SSNID(MCLIRUN8) DTAOPT(*MGTCOL) MGTCOL(PEXDTA/*SSNID)) JOB(ENDPEX) JOBQ(QSYSNOMAX) and press Enter. o This step must complete before moving to the next step. |
Related Information
[{"Type":"MASTER","Line of Business":{"code":"LOB57","label":"Power"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG60","label":"IBM i"},"Platform":[{"code":"PF012","label":"IBM i"}],"Version":"7.1.0"}]
Was this topic helpful?
Document Information
Modified date:
18 December 2019
UID
nas8N1021045