White Papers
Abstract
Overview of trace facility:
The trace facility helps you isolate system problems by monitoring selected system events.
The events can be entry/exit of selected subroutines, kernel routines, interrupt handlers, etc.
When the trace facility is active, the system events are recorded into trace logfile.
There are 8 channels are available to log events, channel 0 is used to record system events.
Problem:
* Any user on the system can start system trace(trace on channel-0) using trace command and
affect system performance at peak hours.
* Any user on the system can generate trace report using trcrpt command and see sensitive
trace data.
Content
Solution:
Restricting both starting and viewing trace facility of system traces (channel-0 trace) from
non-privileged user.
Releases:
AIX 7200-03-00 onwards
AIX 7200-03-00 onwards
How to restrict non-privileged users from starting trace on channel-0?
The restriction can be enabled or disabled by system administrator using "/usr/bin/trcctl -R enable"
and "/usr/bin/trcctl -R disable". By default the restriction is disabled.
The restriction can be enabled or disabled by system administrator using "/usr/bin/trcctl -R enable"
and "/usr/bin/trcctl -R disable". By default the restriction is disabled.
# /usr/bin/trcctl -l
Default Buffer Size: 262144
Default Log File Size: 2621440
Default Log File: /var/adm/ras/trcfile
Non-Root User Buffer Size Maximum: 1048576
Default Components Directory: /var/adm/ras/trc_ct
Default LMT Log Dir: /var/adm/ras/mtrcdir
Restrict non-privileged users from using trace Channel-0: disable <<<<<
Restrict non-privileged users from using trcrpt Channel-0: disable
When the restriction is disabled, any user can start trace on channel-0.
# trace -0 -a
# ps -ef | grep trace | grep -v "grep trace"
root 7340336 1 0 05:01:03 pts/0 0:00 trace -0 -a
\
To enable the restriction
# /usr/bin/trcctl -R enable
# /usr/bin/trcctl -l
Default Buffer Size: 262144
Default Log File Size: 2621440
Default Log File: /var/adm/ras/trcfile
Non-Root User Buffer Size Maximum: 1048576
Default Components Directory: /var/adm/ras/trc_ct
Default LMT Log Dir: /var/adm/ras/mtrcdir
Restrict non-privileged users from using trace Channel-0: enable <<<<<
Restrict non-privileged users from using trcrpt Channel-0: disable
After enabling the restriction, it will prompt error message
if a non-authorized users try to start the trace on channel-0.
$ /usr/bin/trcrpt -0
permission denied: trcrpt on channel-0 is restricted by system administrator. You must have root authority or authorization to trcrpt channel-0 logfile.
How to enable a user to start trace on channel-0?
Make him a privileged user by assigning authorization aix.ras.trace.tracech0
# mkrole authorizations="aix.ras.trace.tracech0" role_tracech0
# chuser roles=role_trcrptch0 testuser
# setkst
Successfully updated the Kernel Authorization Table.
Successfully updated the Kernel Role Table.
Successfully updated the Kernel Command Table.
Successfully updated the Kernel Device Table.
Successfully updated the Kernel Object Domain Table.
Successfully updated the Kernel Domains Table.
Successfully updated the Kernel RBAC log level.
Now the normal user "testuser" can start trace on chennal-0
$ swrole role_tracech0
testuser's Password:
$ trace -a -0
$ ps -ef | grep trace | grep -v "grep trace"
testuser 10158524 1 0 05:17:34 pts/1 0:00 trace -a -0
How to restrict non-privileged users from viewing traces of channel-0?
The restriction can be enabled or disabled by system administrator using "/usr/bin/trcctl -S enable"
and "/usr/bin/trcctl -S disable". By default the restriction is disabled.
# /usr/bin/trcctl -l
Default Buffer Size: 262144
Default Log File Size: 2621440
Default Log File: /var/adm/ras/trcfile
Non-Root User Buffer Size Maximum: 1048576
Default Components Directory: /var/adm/ras/trc_ct
Default LMT Log Dir: /var/adm/ras/mtrcdir
Restrict non-privileged users from using trace Channel-0: disable
Restrict non-privileged users from using trcrpt Channel-0: disable <<<<<
When the restriction is disabled, any user can view traces of channel-0.
# /usr/bin/trcrpt -0 | head -30
Fri Jul 13 05:17:34 2018
System: AIX 7.2 Node: telesto2
Machine: 00F9EA024C00
Internet Address: 097E960F 9.126.150.15
At trace startup, the system contained 4 cpus, of which 4 were traced.
Buffering: Kernel Heap
This is from a 64-bit kernel.
Tracing all hooks.
trace -a -0
ID ELAPSED_SEC DELTA_MSEC APPL SYSCALL KERNEL INTERRUPT
11F 0.000000000 0.000000 setrq: cmd=circularBufferBenchmark pid=10486194 tid=25035009 priority=60 policy=0 rq=0003
100 0.000000986 0.000986 QUEUED INTERRUPT iar=93C0 cpuid=FFFFFFFF
100 0.000001460 0.000474 PROCESSING DEFERRED INTERRUPT i_softpri=0200 previous intpri=0B padj_intpri=0000 lock_intpri=0000 cpuid=FFFFFFFF
11F 0.000003486 0.002026 setrq: cmd=-10551670- pid=10551670 tid=28377541 priority=60 policy=0 rq=0003
200 0.000004818 0.001332 resume circularBufferBenchmark iar=93C0 cpuid=FFFFFFFF
4FB 0.000005375 0.000557 EHCI_T >> (822087946.269066240)aA0e63 In invalid_speed 00 0002 <- S 018F 01A0 errno=416
101E 0.000005750 0.000375 DISP_DONE data=FFFFFFFF tid=01A0018F srad=0000 hrq=0601 trq=0003 hcount=0000 lent=0000 borr=0000 flags=00400000 flags2=00080000
4B0 0.000007281 0.001531 undispatch: old_tid=27263375 CPUID=3
106 0.000008203 0.000922 dispatch: cmd=circularBufferBenchmark pid=10486194 tid=27263375 priority=60 old_tid=27263375 old_priority=60 CPUID=3
101D 0.000008419 0.000216 Home SRAD=0 Home ref1=0 Dispatch SRAD=0 Dispatch ref1=0
101E 0.000008638 0.000219 DISP data=00000001 tid=01A0018F srad=0000 hrq=0601 trq=0003 hcount=0000 lent=0000 borr=0000 flags=00400000 flags2=00080000
4FB 0.000008857 0.000219 EHCI_T << (822087946.269066240)aA0e63 In invalid_speed 00 0001 <- S 018F 01A0 invalid_status
200 0.000009238 0.000381 resume circularBufferBenchmark iar=9408 cpuid=03
...
...
etc...
To enable the restriction
# /usr/bin/trcctl -S enable
# /usr/bin/trcctl -l
Default Buffer Size: 262144
Default Log File Size: 2621440
Default Log File: /var/adm/ras/trcfile
Non-Root User Buffer Size Maximum: 1048576
Default Components Directory: /var/adm/ras/trc_ct
Default LMT Log Dir: /var/adm/ras/mtrcdir
Restrict non-privileged users from using trace Channel-0: enable
Restrict non-privileged users from using trcrpt Channel-0: enable <<<<<
After enabling the restriction, it will prompt error message
if a non-authorized users try to view the traces of channel-0.
$ /usr/bin/trcrpt -0 | head -30
permission denied: trcrpt on channel-0 is restricted by system administrator. You must have root authority or authorization to trcrpt channel-0 logfile.
How to enable a user to view traces of channel-0?
Make him a priviledged user by assigning authorization aix.ras.trace.trcrptch0
# mkrole authorizations="aix.ras.trace.trcrptch0" role_trcrptch0
# chuser roles=role_trcrptch0 testuser
# setkst
Successfully updated the Kernel Authorization Table.
Successfully updated the Kernel Role Table.
Successfully updated the Kernel Command Table.
Successfully updated the Kernel Device Table.
Successfully updated the Kernel Object Domain Table.
Successfully updated the Kernel Domains Table.
Successfully updated the Kernel RBAC log level.
Now the normal user "testuser" can view traces of chennal-0
$ swrole role_trcrptch0
testuser's Password:
$ /usr/bin/trcrpt -0 | head -30
Fri Jul 13 05:17:34 2018
System: AIX 7.2 Node: telesto2
Machine: 00F9EA024C00
Internet Address: 097E960F 9.126.150.15
At trace startup, the system contained 4 cpus, of which 4 were traced.
Buffering: Kernel Heap
This is from a 64-bit kernel.
Tracing all hooks.
trace -a -0
ID ELAPSED_SEC DELTA_MSEC APPL SYSCALL KERNEL INTERRUPT
4B0 0.000000000 0.000000 undispatch: old_tid=3015083 CPUID=2
106 0.000000662 0.000662 dispatch: cmd=j2gt pid=1704296 tid=3015083 priority=63 old_tid=3015083 old_priority=63 CPUID=2
101D 0.000001105 0.000443 Home SRAD=0 Home ref1=0 Dispatch SRAD=0 Dispatch ref1=0
101E 0.000001318 0.000213 DISP data=00000001 tid=002E01AB srad=0000 hrq=0601 trq=0002 hcount=0000 lent=0000 borr=0000 flags=00001000 flags2=00000000
4FB 0.000001523 0.000205 EHCI_T << (822087946.268971520)aA0e63 In invalid_speed 00 0001 <- S 01AB 002E invalid_status
100 0.000001976 0.000453 PROCESSING DEFERRED INTERRUPT i_softpri=0200 previous intpri=0B padj_intpri=0000 lock_intpri=0000 cpuid=02
11F 0.000003705 0.001729 setrq: cmd=-10551670- pid=10551670 tid=28377541 priority=62 policy=0 rq=0002
4FB 0.000005134 0.001429 EHCI_T >> (822087946.268971520)aA0e63 In invalid_speed 00 0002 <- S 01AB 002E errno=46
101E 0.000005376 0.000242 DISP_DONE data=FFFFFFFF tid=002E01AB srad=0000 hrq=0601 trq=0002 hcount=0000 lent=0000 borr=0000 flags=00001000 flags2=00000000
4B0 0.000006384 0.001008 undispatch: old_tid=3015083 CPUID=2
106 0.000007095 0.000711 dispatch: cmd=-10551670- pid=10551670 tid=28377541 priority=62 old_tid=3015083 old_priority=63 CPUID=2 [6 usec]
101D 0.000007298 0.000203 Home SRAD=0 Home ref1=0 Dispatch SRAD=0 Dispatch ref1=0
101E 0.000007525 0.000227 DISP data=00000001 tid=01B101C5 srad=0000 hrq=0601 trq=0002 hcount=0000 lent=0000 borr=0000 flags=00001000 flags2=00000000.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Author: Chetan Gaonkar
Operating System: AIX and VIOS
Hardware: Power
Feedback: aix_feedback@wwpdl.vnet.ibm.com, cgaonkar@in.ibm.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Author: Chetan Gaonkar
Operating System: AIX and VIOS
Hardware: Power
Feedback: aix_feedback@wwpdl.vnet.ibm.com, cgaonkar@in.ibm.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"a8m0z0000001fMuAAI","label":"AIX General Support"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.1.0;7.2.0"}]
Product Synonym
AIX trace facility ;Second failure Data capture; trace restriction;
Was this topic helpful?
Document Information
Modified date:
26 June 2021
UID
ibm16467131