You should read this article if you have used AIX performance tools in AIX 5.1 or older versions, and want to learn about the changes made in 5.2 performance tools. There is a major revamp of performance tools in AIX 5.2. Some existing commands such as schedtune and vmtune have been replaced with the new commands schedo, vmo, and ioo. All of the kernel tuning commands schedo, vmo, ioo, no, and nfso use the same command syntax and provide a consistent behavior.
This article is also for you if you are an experienced UNIX system administrator who understands the basic concepts of kernel internals, but are not familiar with the performance tools available in AIX 5.2. In this part-1 of the article, you will learn:
- The elements of the new 5.2 performance tuning framework which makes it possible to have all the kernel tuning commands work consistently.
- Detailed descriptions of each of the new or modified kernel tuning commands.
The following topics are not within the scope of this article, but there are references provided at the end of this article:
- A description of how the AIX kernel works which helps you interpret the data displayed by these kernel tuning commands.
- An explanation of how performance tuning decisions are made based on the data displayed by performance monitoring tools.
This article is for system administrators who have a need to tune the AIX kernel for their CPU, memory, I/O, or network performance problems. As the first of a series of introduction to AIX 5.2 performance tools, this article focuses primarily on the most commonly used kernel tuning commands. Watch for more articles on other more advanced AIX 5.2 performance tuning facilities.
The features discussed in this article are available in AIX 5.2. Earlier levels of AIX only provide subsets of them. References listed at the end of this article provide you with information applied to AIX 5.2.
The following AIX 5.2 filesets need to be installed to get these features:
- bos.perf.tune: to get the schedo, vmo, and ioo commands
- bos.net.nfs.client: to get the nfso command
- bos.net.tcpip.client: to the get no command
Prior to AIX 5.2, the commands provided to tune the kernel are vmtune, schedtune, no, and nfso. These commands have the following limitations:
- Not all commands have an option to reset parameters to default values
- The range and parameter values are not available
- No validation checking for the new values
- Inconsistent command parameters
- No clean way to make persistent changes (changes to the /etc/rc files are required in order for the changes to persist across boots)
Enhancements in AIX 5.2 to overcome these shortcomings are:
- Some new commands are added and some existing commands are enhanced, so that a set of consistent commands are available to manipulate kernel parameters.
- SMIT panels are also provided for these commands so that you do not have to remember command options and valid range of parameters. These menus can be reached using smitty tuning.
- The parameter values are now stored in stanza files in the directory /etc/tunables, so that, for example, parameter settings can be restored to their original values, and applied as permanent changes.
With these enhancements, the performance analysis and tuning tools are greatly improved in AIX 5.2.
In AIX 5.2, vmo and ioo are the two new commands that replace vmtune, and schedo replaces schedtune. The ioo command handles all the I/O related tuning parameters, while the vmo command handles all the other VMM parameters previously managed by vmtune. All existing parameters are covered by the new commands.
| Before AIX 5.2 | AIX 5.2 | Tunables handled |
| vmtune | vmo | Virtual memory tuning parameters |
| vmtune | ioo | I/O related tuning parameters |
| schedtune | schedo | CPU scheduler tunable parameters |
The bos.adt.samples directory in AIX 5.2 still contains the vmtune and schedtune commands, but they are only compatibility shell scripts calling vmo, ioo, and schedo as appropriate. The compatibility scripts only support changes to parameters which can be changed interactively. Parameters that need bosboot and then require a reboot of the machine to be effective are no longer supported by the vmtune script. You must use the new command vmo -r to change those parameters.
The no and nfso commands have been enhanced to support making permanent changes to tunable parameters. They now interact with the /etc/tunables/nextboot file to achieve this new functionality. They both also have a new -h flag which can be used to display help about any parameter. The content of the help includes the purpose of the parameter, the possible values (default, range, and type), and diagnostic and tuning information to decide when to change the parameter value. This information is also listed entirely in the respective man pages.
All the tunable parameters manipulated by the tuning commands (no, nfso, vmo, ioo, and schedo) have been classified into seven categories:
| Category | Type Abbreviation | Properties | Example |
| Dynamic | D | Parameter can be changed at any time. | timeslice parameter of schedo: the largest number of clock ticks that a thread can be in control before facing the possibility of being replaced by another thread. |
| Static | S | Parameter can never be changed (read-only parameter). | memory_frames parameter of vmo: the number of real-memory page frames in the system |
| Reboot | R | Parameter can only be changed during the reboot. | ipqmaxlen parameter of no: the number of received packets that can be queued on the IP protocol input queue. |
| Bosboot | B | The parameter can only be changed by running bosboot followed by a reboot of the machine. | mempools parameter of vmo: the number of memory pools that the system real memory is split into. |
| Connect | C | The parameter only apply to future socket connections, changes of this type of parameter automatically restarts inetd. | rfc1323 parameter of no: Enables window scaling and timestamps as specified by RFC 1323 (TCP Extensions for High Performance). So that TCP window sizes (tcp_recvspace and tcp_sendspace) can be larger than 64KB. |
| Mount | M | The parameter changes are only effective for future file systems or directory mounts | nfs_v3_pdts parameter of nfso: the number of tables for memory pools used by the biods for NFS Version 3 mounts. |
| Incremental | I | Incremental the parameter can only be incremented, except at boot time | psetimers parameter of no: the maximum number of timers to allocate by streams. |
The man page for each of the five tuning commands contains the complete list of all the parameter manipulated by each of the commands and for each parameter, its type, range, default value, and any dependencies on other parameters.
All of the five tuning commands (no, nfso, vmo, ioo, and schedo) now interact with files in /etc/tunables. Three files under /etc/tunables have special names and meaning:
| nextboot | This file contains all the tunable parameter values to be applied to the next reboot. This file is automatically applied at boot time. The bosboot command also get the value of Bosboot types tunables from this file. It contains all tunable settings made permanent. |
| lastboot | This file is automatically generated at boot time. It contains the full set of tunable parameters, with their values after the last boot. Default values are marked with # DEFAULT VALUE. |
| lastboot.log | This should be the only file in /etc/tunables that is not in the stanza format. It is automatically generated at boot time, and contains the logging of the creation of the lastboot file, that is, any parameter change made is logged. Any change which could not be made (possible if the nextboot file was created manually and not validated with tuncheck) is also logged. |
Except for the lastboot.log file, this directory only contains ASCII stanza files that contain parameter=value pairs. The Tunables files currently support 6 different stanzas: one for each of the tunable command (schedo, vmo, ioo, no, and nfso), plus a special info stanza. The five stanzas schedo, vmo, ioo, no and nfso contain tunable parameters managed by the corresponding command. The info stanza is used to store information about the purpose of the tunable file and the level of AIX on which it was validated.
The following is a sample tunables file:
info:
Description = "Set of tunables for departmental server"
AIX_level = "5.2.0.0"
Kernel_type = "UP"
Last_validation = "2002-06-16 12:11:11 CDT current"
schedo:
timeslice = "2" # set timeslice to 30ms
sched_D = "DEFAULT" # value was 123
vmo:
minperm = "48538"
memory_frames = "65536" # STATIC (never restored)
ioo:
iotunable = "value"
no:
ipforwarding = "1"
ipsrcrouteforward = "1"
thewall = "STATIC" # value was 131072 (never restored)
nfso:
nfs_allow_all_signals = "0" # DEFAULT VALUE
nfs_device_specific_bufs = "0"
|
Five new commands are provided in AIX 5.2 bos.perf.tune fileset to modify the tunable files:
| Command | Function | Comment |
| tuncheck | Validates a file in current context or next boot context, check ranges, dependencies and prompts to run bosboot if necessary. | If you create a tunable file with an editor, or copying a file from another machine, you must run the tuncheck command to validate it. |
| tundefault | Resets all tunable parameters to their default values, can be applied on current or reboot values ( -r flag ) | |
| tunsave | Saves all current values to a file including the nextboot file. | |
| tunchange | Updates stanza in tunable files. | This command unconditionally changes the stanza without validating the parameter. Use this with caution. |
| tunrestore | Applies values from a file, or at the nextboot ( -r flag ). With -r, the command validates the file and copies it over to the current nextboot file. |
Tunable files can be created and modified using one of the following options:
- Using smit or Web-based System Manager
- Using an editor
- Copied from another machine
- Using the tuning commands (vmo, ioo, schedo, no, or nfso)
- Using the tunsave command to create or overwrite files in the /etc/tunables
- Using the tunrestore -f command to update the nextboot file
All five tuning commands (ioo, nfso, no, vmo, and schedo) now use the following common syntax.
command [-p|-r] {-otunable[=newvalue]}
command [-p|-r] {-dtunable}
command [-p|-r-D
command [-p|-r-a
command-h [tunable]
command-L [tunable]
command-xtunable]
Note: Multiple -o, -d, -x, and -L are allowed.
| Flag | Description |
| -a | Displays the values for all tunable parameters. |
| -h | Displays command help or displays help about a specific tunable parameter. |
| -d | Resets tunables to default values. |
| -D | Resets all tunables to their default values. |
| -o | Sets tunable to specific value. |
| -p | Makes changes to both current and reboot values, that is, turns on the updating of the /etc/tunables/nextboot file in addition to the updating of the current value. This flag cannot be used on Reboot or Bosboot type parameters because their current value cannot be changed. |
| -r | Make changes to reboot values, that is, turns on the updating of the /etc/tunables/nextboot file. If any parameter of type Bosboot is changed, you will be prompted to run bosboot. |
| -L | Displays the characteristics of one or all tunables, one per line. |
| -x | Displays the characteristics of one or all tunables, one per line, in a comma separated format suitable for loading into spreadsheet. |
Use the -L flag to display parameter characteristics
The -L flag lists the current and reboot value, range, unit, type, and dependencies of all tunables parameters managed by the command.
> schedo -L | more
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
value value value value value
v_repage_hi 0 0 0 0 2147483647 D
v_repage_proc 4 4 4 0 2147483647 D
v_sec_wait 1 1 1 0 2147483647 seconds D
v_min_process 2 2 2 0 2147483647 processes D
v_exempt_secs 2 2 2 0 2147483647 seconds D
pacefork 10 10 10 10 2147483647 clock ticks D
sched_D 16 16 16 0 32 D
sched_R 16 16 16 0 32 D
timeslice 1 1 1 0 2147483647 clock ticks D
maxspin 16384 16384 16384 1 4294967295 spins D
%usDelta 100 100 100 0 100 D
affinity_lim 7 7 7 0 100 dispatches D
idle_migration_barrier 4 4 4 0 100 sixteenth D
fixed_pri_global 0 0 0 0 1 boolean D
> schedo -L maxspin
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
value value value value value
maxspin 16384 16384 16384 1 4294967295 spins D
|
Try to use -L flag for yourself with the other four tuning commands:
- vmo -L | more
- ioo -L | more
- no -L | more
- nfso -L | more
Use the -x flag to display parameter characteristics, one per line, in a comma separated format
> vmo -x memory_frames,786432,,786432,,,4KB pages,S, pinnable_frames,740146,,740146,,,4KB pages,S, maxfree,128,128,128,16,204800,4KB pages,D,minfree memory_frames minfree,120,120,120,8,204800,4KB pages,D,maxfree memory_frames minperm%,20,20,20,1,100,% memory,D,maxperm% minperm,149033,,149033,,,,S, maxperm%,80,80,80,1,100,% memory,D,minperm% maxclient% maxperm,596137,,596137,,,,S, strict_maxperm,0,0,0,0,1,boolean,D, maxpin%,80,80,80,1,99,% memory,D,pinnable_frames memory_frames maxpin,629146,,629146,,,,S, maxclient%,80,80,80,1,100,% memory,D,maxperm% lrubucket,131072,131072,131072,65536,,4KB pages,D, defps,1,1,1,0,1,boolean,D, nokilluid,0,0,0,0,2147483647,uid,D, numpsblks,131072,,131072,,,4KB pages,S, npskill,1024,1024,1024,1,131071,4KB pages,D, npswarn,4096,4096,4096,0,131071,4KB pages,D, v_pinshm,0,0,0,0,1,boolean,D, pta_balance_threshold,50,50,50,1,99,% pta segment,D, pagecoloring,0,0,0,0,1,boolean,B, framesets,2,2,2,1,10,,B, mempools,2,1,2,1,2,,B, lgpg_size,0,0,0,0,268435456,bytes,B,lgpg_regions lgpg_regions,0,0,0,0,,,B,lgpg_size num_spec_dataseg,n/a,0,0,0,,,B, spec_dataseg_int,n/a,512,512,0,,,B, memory_affinity,n/a,1,1,0,1,boolean,B, |
Use the -h flag to display command help or help about a specific tunable parameter
> vmo -h
Usage: ioo -h [tunable] | {-L [tunable]} | {-x [tunable]}
ioo [-p|-r] (-a | {-o tunable})
ioo [-p|-r] (-D | ({-d tunable} {-o tunable=value}))
-h Display help about the command and its arguments
-h tunable Display help about a tunable
-L [tunable] List information about one or all tunables in a
table format
-x [tunable] List information about one or all tunables in a
comma-separated format
-a Display value for all tunables, one per line
-o tunable Display current value of a tunable
-D Reset all tunables to their default values
-d tunable Reset tunable to its default value
-o tunable=value Set tunable to value
-r Make change(s) (-D/-d/-o) or display (-a/-o) apply to
nextboot value
-p Make change(s) (-D/-d/-o) or display (-a/-o) apply to
permanent (current and nextboot) value
> vmo –h minfree
Help for tunable minfree:
Specifies the minimum number of frames on the free list at which the VMM starts
to steal pages to replenish the free list. Default: maxfree - 8; Range: 8 to 204
800. Page replacement occurs when the number of free frames reaches minfree. If
processes are being delayed by page stealing, increase minfree to improve response
time. The difference between minfree and maxfree should always be equal to or
greater than maxpgahead.
> schedo –h sched_D
Sets the short term CPU usage decay rate. Default: 16; Range: 0 to 32.
The default is to decay short-term CPU usage by 1/2 (16/32) every second.
Decreasing this value enables foreground processes to avoid competition with
background processes for a longer time.
|
Use the –o tunable=value option to change the value of a tunable, and the -d option to reset a tunable to its default value
Example 1: schedo command
> schedo -L sched_R
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
Value value value value value
sched_R 16 16 16 0 32 D
> schedo -o sched_R=19
Setting sched_R to 19
> schedo -L sched_R
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
Value value value value value
sched_R 19 16 16 0 32 D
> schedo -d sched_R
Setting sched_R to 16
> schedo -L sched_R
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
Value value value value value
sched_R 16 16 16 0 32 D
|
Example 2 : ioo command
> ioo –o sync_release_ilock=1
Setting sync_release_ilock to 1
> ioo -o minpgahead=4
Setting minpgahead to 4
> ioo -L sync_release_ilock
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
value value value value value
sync_release_ilock 1 0 0 0 1 boolean D
> ioo -L minpgahead
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
value value value value value
Minpgahead 4 2 2 0 4096 4KB pages D maxpgahead
> ioo -D
Setting minpgahead to 2
Setting maxpgahead to 8
Setting pd_npages to 65536
Setting maxrandwrt to 0
Setting numclust to 1
Setting numfsbufs to 186
The numfsbufs change is only effective for future mounts
Setting sync_release_ilock to 0
Setting lvm_bufcnt to 9
Setting j2_minPageReadAhead to 2
Setting j2_maxPageReadAhead to 8
Setting j2_nBufferPerPagerDevice to 512
The j2_nBufferPerPagerDevice change is only effective for future mounts
Setting j2_nPagesPerWriteBehindCluster to 32
Setting j2_maxRandomWrite to 0
Setting j2_nRandomCluster to 0
Setting hd_pbuf_cnt to 320
|
To test boundaries, types, and dependencies on current values
Warning or error messages are displayed whenever tunable values are not within boundaries.
> nfso -o nfs_max_threads=4 nfso: 1831-535 Error occurred trying to set nfs_max_threads to 4. > ioo -o minpgahead=5120 invalid tunable value 5120 maximal value for tunable minpgahead is 4096 > schedo -o pacefork=9 invalid tunable value 9 minimal value for tunable pacefork is 10 > vmo -o pagecoloring=1 Tunable pagecoloring is of type B and can't be changed without -p or -r |
Use the -p flag to change permanent values
The -p flag is used in combination with -o, -d or -D and makes changes that apply to both the current and reboot values. It turns on the updating of the /etc/tunables/nextboot file in addition to updating the current value.
> ioo -L lvm_bufcnt
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
value value value value value
lvm_bufcnt 9 9 9 1 64 128KB/buffer D
> cat /etc/tunables/lastboot | fgrep lvm_bufcnt
lvm_bufcnt = "9"
> cat /etc/tunables/nextboot | grep lvm_bufcnt
lvm_bufcnt = "9"
> ioo -p -o lvm_bufcnt=12
Setting lvm_bufcnt to 12 in nextboot file
Setting lvm_bufcnt to 12
> ioo -L lvm_bufcnt
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
value value value value value
lvm_bufcnt 12 9 12 1 64 128KB/buffer D
> cat /etc/tunables/lastboot | fgrep lvm_bufcnt
lvm_bufcnt = "9"
> cat /etc/tunables/nextboot | grep lvm_bufcnt
lvm_bufcnt = "12"
|
Note that both the current and reboot values have changed when -p was specified. The /etc/tunables/nextboot file also changes, but the /etc/tunables/lastboot does not.
Use the -r flag to change reboot value.
The -r flag is used in combination with -o, -d, or -D and makes changes that apply to reboot values. For example, it turns on the updating of the /etc/tunables/nextboot file. If any parameter of type Bosboot is changed, you will be prompted to run bosboot.
> vmo -L mempools
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
Value value value value value
Mempools 1 1 1 1 2 B
> cat /etc/tunables/lastboot | fgrep mempools
mempools = "1"
> cat /etc/tunables/nextboot | grep mempools
mempools = "1"
>vmo -p -o mempools=2
vmo: 1485-115 Tunable mempools is of type B and cannot be changed without –r
> vmo -r -o mempools=2
Setting mempools to 2 in nextboot file
Warning: bosboot must be called and the system rebooted for the mempools change to take effect
Run bosboot now? [y/n] y
bosboot: Boot image is 16773 512 byte blocks.
Changes will take effect only at next reboot
> vmo -L mempools
Name Current Default Reboot Minimum Maximum Unit Type Dependencies
Value value value value value
Mempools 1 1 2 1 2 B
> cat /etc/tunables/lastboot | fgrep mempools
mempools = "1"
> cat /etc/tunables/nextboot | grep mempools
mempools = "2"
> no -r -o rto_limit=10
> no -L rto_limit
Name Value Default Boot Min Max Unit Type
rto_limit 7 7 10 1 64 roundtriptime D
|
Note that the reboot value has changed when -p was specified, but the current value stays the same. The /etc/tunables/nextboot file also changes, but the /etc/tunables/lastboot does not.
Use the tunsave command to hand created file
After verify the mempools, nfs_v2_bufs and rto_linit are listed in the nextboot file, you can use the tunsave command to create myfile.
/etc/tunables:> tunsave -f myfile |
The myfile shows a list of all parameters currently not set to the default values
Use the tuncheck command to validate a tunable file
/etc/tunables:> cat myfile
info:
Description = "tunsave -f myfile"
AIX_level = "5.2.0.0"
Kernel_type = "MP"
Last_validation = "2003-06-02 09:33:06 CDT (current, reboot)"
schedo:
vmo:
mempools = "2"
ioo:
lvm_bufcnt="12
no:
rto_limit = "10"
nfso:
nfs_v2_vm_bufs = "2000"
/etc/tunables:> tuncheck -f myfile
Tunable mempools is of type B and can't be changed without -p or -r
Checking failed
Messages should have been provided
/etc/tunables> tuncheck -r -f myfile
Setting mempools to 2 in nextboot file
Warning: bosboot must be called and the system rebooted for the mempools change to take effect
Run bosboot now? [y/n] y
bosboot: Boot image is 16773 512 byte blocks.
Changes will take effect only at next reboot
|
Use the tunrestore command to apply tunable values from a file
/etc/tunables:> cat myfile
info:
Description = "tunsave -f myfile"
AIX_level = "5.2.0.0"
Kernel_type = "MP"
Last_validation = "2003-06-02 09:33:06 CDT (current, reboot)"
schedo:
vmo:
mempools = "2"
ioo:
lvm_bufcnt="12
no:
rto_limit = "10"
nfso:
nfs_v2_vm_bufs = "2000"
/etc/tunables:> tunrestore -r -f myfile
/etc/tunables:> shutdown –Fr
|
After rebooting, check the lastboot and lastboot.log file.
/etc/tunables: > cat lastboot | grep mempools mempools = "2" /etc/tunables: > cat lastboot | grep rto_limit rto_limit = "10" /etc/tunables: > cat lastboot | grep nfs_v2_vm_bufs nfs_v2_vm_bufs = "2000" |
To ease the migration and provide compatibility to customers who are migrating to AIX 5.2 from a previous version of AIX, a pre-5.2 compatibility mode is provided. This mode is controlled by a new pre520tune sys0 attribute.
To check the current setting:
lsattr –E –l sys0 | grep pre52tune |
To change the setting:
chdev –l sys0 –apre520tune=disable|enable |
The pre520tune setting can also be displayed and changed using SMIT chgsys or Websm.
The pre520tune mode is automatically enabled when migrating from 5.1, and disabled for a new installation of AIX 5.2. When running in compatibility mode, permanent tunable parameter settings are not set by applying values from the /etc/tunables/nextboot file, and can still be set by embedding calls to tuning commands in scripts called during the boot process. The following will be observed in compatibility mode:
- The current behavior of the tuning commands, with the exception of the vmtune parameters mentioned in the Overview section, is completely preserved. (The vmtune compatibility script only support changes to parameters which can be changed interactively. That is, parameters that need bosboot and then require a reboot of the machine to be effective are no longer supported by the vmtune script.)
- The no command allows Reboot parameters to be changed without the -r flag/
- Values from the nextboot file are not applied during boot.
- Last boot and the lastboot.log are created. The lastboot.log file contains only a warning that AIX is currently running in compatibility mode and that the nextboot file has not been applied.
- No difference for bosboot parameters
Switching to the non-compatibility mode while preserving the current reboot settings can be done by first changing pre520tune, and then by running the following command:
/etc/tunables> tunrestore -r -f lastboot |
which will copy the content of the lastboot file to the nextboot file.
- AIX 5L Version 5.2 Performance Tools Guide and Reference http://publib16.boulder.ibm.com/doc_link/en_US/a_doc_lib/aixbman/prftools/prftoolstfrm.htm
- AIX 5L Version 5.2 Performance Mangement Guide http://publib16.boulder.ibm.com/pseries/en_US/aixbman/prftungd/prftungdtfrm.htm
Nam Keung works as a Senior Programmer at IBM. Nam works in the area of AIX communication development, AIX multimedia, SOM/DSOM development, and Java performance. His current assignment involves helping Independent Software Vendors (ISVs) in application design, deploying applications, performance tuning, and education for the pSeries platform. You can contact Nam at namkeung@us.ibm.com.
Lee Cheng works as a senior consultant for RS/6000 and AIX software vendors. She provides support to them in the areas of application benchmarks, performance tuning, application porting, and internationalization. Before joining the RS/6000 ISV Technical Support group, she was a developer for compilers and the AIX system management component. She holds a M.S. degree in Computer Science from the University of Kentucky. Her publications includes AIX Performance Tuning: CPU usage, AIX Performance Tuning: Focus on Memory. You can contact Lee at chenglc@us.ibm.com.
Wayne Huang is a senior consultant for IBM eServer pSeries and AIX systems with a focus on e-business, banking, finance, and securities industries. He provides AIX support to ISVs in the areas of application design, problem determination, system performance tuning, and application benchmarks. He holds a BS in Physics from National Taiwan University and an MS in Computer Science from the University of Texas at Austin. You can reach him at huangw@us.ibm.com.
Table of contents
Local resources
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
