Before we look further in to memory affinity we need to recap on the scheduling of processes and process threads of a multi-threaded process to simultaneous multi-threading (SMT) processors like POWER7.  POWER5 and POWER6 had two modes of SMT off (one thread) and on (two threads) - with SMT=on two processes run at the same time (in the same clock cycle) on the CPU-core but using the different logical units inside of the CPU-core (units like the integer maths (there is more than one), floating point maths (there is more than one), compare and... [More]

Tags:  logical core powervm vp aix smt power7 processor power

 This mysterious AIX CPU Folding area is often misunderstood, so below is what I know from osmosis from talking to various guru level developers over the last 10 years. Shared Processor virtual machines (LPARs for the old fashioned) have a setting called Virtual Processors (or VP for short). This is the number of physical CPUs that the virtual machines can spread out across - in fact, I prefer to call it the "spreading factor" as it is much more obvious what it means.  This can be the upper threshold for the number of CPUs that can be... [More]

Tags:  power7 powervm power6 hypervisor aix folding aix6 aix7 virtual processor

I am preparing sessions for a POWER/AIX Technical Conference in Norway - unfortunately the only place they could book for the event was a Ski Resort up in the mountains and there is still good snow. It is a tough gig but someone has to do it :-)  As part of that conference, I am updating them on Systems Director and demonstrating it. I had a slide with these hints and tips. A slide with zero percent marks for style (far to many words and just a long list) and I thought ... I should share these with everyone, so here they are. Most of these are... [More]

Tags:  vios power7 aix endpoint systems nim director server power

The title should read "Local, Near & Far ..."  - I will not correct it or links might fail.  With a shared processor virtual machine (I am calling this "VM" but was called LPAR!) there are various suggestions of setting Entitlement ("Desired processing units" on the LPAR profile on the HMC, I am calling this "E") and Virtual Processor numbers (I am calling this "VP"). For Capped, the Entitlement is the maximum guaranteed CPU time that you can't go over and you round up the Entitlement to the... [More]

Tags:  processor virtual powervm power7 aix low power6 entitlement

Update in 2016: Please note this blog is from 2011 during the initial POWER7 days and technology has moved on with faster CPUs and memory in addition there has been software improvements.  I am amazed how many good computer people read this and assume this blog is true to all time! Virtual Ethernet is faster now but there is also a warning here. Here is an analogy. Most vehicles can do 10 MPH (including me on a bike), most cars can do 100 MPH (including my family car) but very few vehicles can do 1000 MPH. That last times ten multiplier is... [More]

Tags:  vios machine misunderstood power6 power7 aix virtual ethernet speed powervm

So you know about Power7 Local, Near and Far memory for your actual machine but what is your Virtual Machine (LPAR) actually using? There are three key commands to show you (lssrad, mpstat and topas) and we will look at some example output.   First, we need to define an SRAD or a Scheduler Resource Affinity Domain .  If you have used Resource Sets with AIX WLM or WPAR then you have a good idea what these are like. An SRAD is a group of resources but in our case CPU/cores and the associated memory that is directly attached to it. As an... [More]

Tags:  lssrad topas affinity srad mpstat powervm aix6 power7 resource aix7 domain scheduler

POWER Relative Performance (rPerf) is often used as a way to approximate the expected difference in performance between two Power Systems servers. Although rPerf is a useful tool, it is important to understand the limitations of using rPerf  to provide an estimate the performance of your specific workloads in your particular environment with a new server, First, rPerf numbers, like any published benchmark, represent the best case result when the application, configuration and system resources are all optimized -- factors that  are likely not... [More]

Tags:  aix powervm rperf lpar sizing power7

  I got another WPAR question today: From the Global AIX, I can add a filesystem to /wpars/WPARname/directory so the WPAR has access but what if I don't want to have the filesystem mounted there in the Global AIX?   For example: I have a Global AIX filesystem called /tools that I want to mount in to one or many WPARs and appear and /temp in the WPAR(s)?   My first answer is well you can use NFS because I do that all the time and I am a heavy user of NFS - no jokes please. I use NFS heavily!  If the NFS server exports the mount point for every... [More]

Tags:  - filesystem aix global workload sharing wpar partition power7 aix6

I thought I should summarise the long eleven part Local, Near & Far POWER7 Affinity series.   1) Placement: Find out the layout of your boxes CPU and RAM and if the RAM is evenly distributed across available DIMMs Find out the placement of your Virtual Machines (LPARs) with lssrad -av  - or - topas -M 2) SMT4 : Expect POWER7 SMT4 CPU use to “look” different POWER5 & 6 have two equal threads POWER7 shuts down threads 3 & 4 and even thread 2 - when there is not enough processes running. 3) Entitlement : Only set minimum Entitlements,... [More]

Tags:  lpar aix hypervisor entitlement processor systems firmware power power7 virtual powervm

I just realised that I got help from the AIX developers to explain the output of the mpstat -d command for my POWER7 and Affinity Technical University sessions in Miami and Copenhagen that I never passed on to every one.  In an earlier blog we examined some mpstat columns but skipped others.   The nearly undocumented mpstat stats are: S0rd, S1rd, S2rd, S3rd, S4rdand S5rd. In the AIX manuals it states: S0rd = The percentage of thread re-dispatches within the scheduling affinity domain 0.  And likewise for the other numbers - but what does... [More]

Tags:  cpu power7 affinity virtual physical logical

I have been wondering why the lower end POWER7 machines have local and far memory and not local and near. Perhaps you wondered too! Well at the Miami Power Technical University, I got to talk to Dr Joel Tendler (IBMer) and a POWER7 processor guru and put the question to him. He covered this sort of architectural topic in his presentation at the event and I learnt a lot in this area by listening to the "master". Below is some background and the explanation too.   The POWER7 chips has two memory controllers for maximum performance but... [More]

Tags:  795 xyz 780 near systems aix memory ab 770 far power7 power local bus

N-Port Id Virtualisation (NPIV) and virtual Fibre Channel adapters - like high speed physical adapters (see an earlier blog) do require memory to operator at full speed.  NPIV effectively turns the Virtual I/O Server (VIOS) into a kind of virtual SAN switch where packets are passed through with no changes from the physical adapter to the client Virtual Machine (LPAR).  I found it very hard to find a recommendation in the documentation but was given a rule of thumb.  As before it should be noted that starving the VIOS and the Hypervisor of... [More]

Tags:  thumb power7 of rule npiv adapters power aix virtual

I was just asked this question by customer and it got me thinking, Items I would include: WPAR takes seconds to create and LPARs minutes LPARs need setting VIOS LV or LUNs but WPARs add simpler NFS mount points, or can just use Global AIX diskspace options LPAR needs 512 to 1GB to boot AIX and a WPAR takes just ~60 MB (yes sixty megabytes) You can share application code say 1 GB in each and every LPAR (40 LPAR = 40 GB) or just one shared read-only copy for all WPARs 40 WPAR = 1 GB).This saves man-power in maintenance, disk space AND memory (if... [More]

Tags:  lpar power7 aix7 aix wpar 5.2

It has bee a long few weeks traveling to Dublin, Las Vegas, Slovenia, Germany and Sweden to give lots of presentations.  Six Weeks and five countries and the expenses are piling up on my desk. I was presenting a core set of topics like POWER7+ announcement summary, VIOS Shared Storage Pools (SSP3), Workload Partitions (WPAR) and Versioned WPARs to run AIX 5.2 and AIX 5.3, quick introduction to PowerSC (security tools), a Techie "what you need to know about" Systems Director for AIX, and Quick Dips in to Active Memory Deduplication... [More]

Tags:  ssp pools shared isd storage aix power director wpar systems aso dso powersc power7