Lenovo: SMX enable on 8 socket systems can hang - Lenovo System x3950 X6 (6241), x880 X6 Compute Node (7196)

Troubleshooting

Problem

Linux kernels used in VMware are sensitive to 8 socket boot while SMX (TXT) is enabled. Each Linux kernel behaves slightly differently. RHEL6.5 does boot 8S with SMX enabled. RHEL6.5 does boot 4S with SMX disabled. RHEL6.5 does boot 8S with SMX disabled. Using 2Q2015 UEFI release A9E122XUS VMware5.5 hangs during boot with 8S and SMX enabled If SMX is disabled, each kernel boots to the Operating System (OS) as expected. (where TXT = Intel Trusted eXecution Technology, SMX = Safer Mode Extensions)

Resolving The Problem

Source

RETAIN tip: H214637

Symptom

Linux kernels used in VMware are sensitive to 8 socket boot while SMX (TXT) is enabled. Each Linux kernel behaves slightly differently.

RHEL6.5 does boot 8S with SMX enabled.
RHEL6.5 does boot 4S with SMX disabled.
RHEL6.5 does boot 8S with SMX disabled.
Using 2Q2015 UEFI release A9E122XUS

VMware5.5 hangs during boot with 8S and SMX enabled

If SMX is disabled, each kernel boots to the Operating System as expected.

(where TXT = Intel Trusted eXecution Technology, SMX = Safer Mode Extensions)

Affected configurations

The system may be any of the following IBM servers:

Lenovo Flex System x880 Compute Node, Type 7196, any model, any x880
Lenovo System x3950 X6, Type 6241, any mode

This tip is not software specific.

This tip is not option specific.

The following system BIOS or uEFI level(s) are affected:

lnvgy_fw_uefi_a9e112d-1.00_anyos_32-64

The system is configured with 8 CPU's.

Solution

The fix wil be an update to the UEFI code.

The target date for this release is scheduled for Forth Quarter 2015.

The file is or will be available by selecting the appropriate Product Group, type of System, Product name, Product machine type, and Operating system on IBM Support's Fix Central web page, at the following URL:Â Â Â Â

http://www.ibm.com/support/fixcentral/

Workaround

Not all Linux kernels have the same sensitivity.

VMware 5.5 in particular appears to be most sensitive.

The workarounds include:

Disable SMX (Trusted Execution Technology) in UEFI F1 setup menu.

Partition to be two (2) 4S systems with SMX enabled.

Additional information

An example of the code impact is listed below.
TxtDxeLib.c is part of the UEFI open source. The typecast to UINT8 effectively removes the ApicId required to have more than 256 cores.

With SMX disabled, users can Secure Shell (SSH) to VMware to validate:

	approximately # bootOptionÂ -o Options : vmbTrustedBoot=false tboot=0x0x101b000 no-auto-partition bootUUID=ea25656283f7dd097b910b14197b16c8 When SMX enabled you can SSH to VMware approximately # bootOptionÂ -o Options : vmbTrustedBoot=true tboot=0x0x101b000 no-auto-partition bootUUID=ea25656283f7dd097b910b14197b16c8 Â ------------ TxtDxeLib.c ------------ // // Get All processor APIC ID info // for (CpuIndex = 0; CpuIndex < CpuCount; CpuIndex ++){ Status = LtDxeCtx->MpService->GetProcessorInfo ( LtDxeCtx->MpService, CpuIndex, &ProcInfo); ASSERT_EFI_ERROR (Status); LtDxeCtx->ApicId[CpuIndex] = (UINT8)ProcInfo.ProcessorId; ^^^^^ DEBUG ((EFI_D_INFO, "TXT-LIB APIC[%d] = 0x%08x\n", CpuIndex, LtDxeCtx->ApicId[CpuIndex])); } }

In this case the typecast to UINT8 lost the pertinent information for the sockets in question. There are other instances within the code which also do not handle 8S with SMX enabled.

UEFI Open Source code has some issues with 8S which both Lenovo and Intel are investigating. IPS 6000093743 has been opened to address this issue.

Document Location

Worldwide

Operating System

Lenovo x86 servers:Operating system independent / None

[{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QUOFMOF","label":"Lenovo x86 servers->Lenovo System x3950 X6->6241"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QUOFNB5","label":"Lenovo x86 servers->Lenovo x880 X6 Compute Node->7196"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}}]

Tips