Topic
14 replies Latest Post - ‏2012-02-15T16:28:03Z by adrianfrederic
SystemAdmin
SystemAdmin
2404 Posts
ACCEPTED ANSWER

Pinned topic A nmon problem wtih errno=13

‏2008-04-25T07:27:31Z |
hi,

I have a problem whith nmon at aix;

when we run it with no root it have a problem like this,and when we
run as root it seems good!

-bash-3.00$ nmon

AIX version 5.3.0.0 and starting up nmon nmon_aix53
./nmon_aix53
ERROR: Assert Failure in file="nmon11.c" in function="main" at line=3300
ERROR: Reason=System call returned -1
ERROR: Expression=
ERROR: errno=13
ERROR: errno means : Permission denied

-bash-3.00$ oslevel -s
5300-04-03

how can i do that? thanks!

steven
Updated on 2012-02-15T16:28:03Z at 2012-02-15T16:28:03Z by adrianfrederic
  • nagger
    nagger
    1594 Posts
    ACCEPTED ANSWER

    Re: A nmon problem wtih errno=13

    ‏2008-04-25T08:20:12Z  in response to SystemAdmin
    Can you tell us the exact version of nmon you are running?

    Use: nmon -?
    and you should get the line like
    Version: nmon11e v11e

    Also are you absolutely sure the root user and regular use is starting the same binary file.
    With different $PATH settings it is easy to assume typing nmon starts the same program when it is not.
    Try: whence nmon
    in both root and regular user - this assumes ksh.

    The error line (if this is nmon11e) is a libperfstat library call to perfstat_diskadapter() to get adapter information. Is there anything "tricky" with your disks or disk adapters?

    Can you paste a copy of the lsconf output?

    nmon works fine on my machine level as a regular user: oslevel -s
    5300-04-00

    Get back to us with some answers and we can investigate further, thanks Nigel
    • SystemAdmin
      SystemAdmin
      2404 Posts
      ACCEPTED ANSWER

      Re: A nmon problem wtih errno=13

      ‏2008-04-25T08:45:01Z  in response to nagger
      -bash-3.00$ nmon -?

      AIX version 5.3.0.0 and starting up nmon nmon_aix53
      ./nmon_aix53 -?

      Hint: nmon_aix53 -h -s <seconds> -c <count> -f -d -t -r <name> -x
      Version: nmon_aix53 v11e
      -h FULL help information - much more than here
      Interactive-Mode:
      read startup banner and type: "h" once it is running
      For Data-Collect-Mode (-f)
      -f spreadsheet output format note: default -s300 -c288
      optional
      -s <seconds> between refreshing the screen default 2
      -c <number> of refreshes default millions
      -t spreadsheet includes top processes
      -x capacity planning (15 min for 1 day = -fdt -s 900 -c 96)

      regular user :
      $ whence nmon
      /usr/bin/nmon
      root
      1. whence nmon
      /usr/bin/nmon

      1. lsconf
      System Model: IBM,9131-52A
      Machine Serial Number: 065FFDG
      Processor Type: PowerPC_POWER5
      Number Of Processors: 4
      Processor Clock Speed: 1648 MHz
      CPU Type: 64-bit
      Kernel Type: 64-bit
      LPAR Info: 1 06-5FFDG
      Memory Size: 7936 MB
      Good Memory Size: 7936 MB
      Platform Firmware level: SF240_284
      Firmware Version: IBM,SF240_284
      Console Login: enable
      Auto Restart: true
      Full Core: false

      Network Information
      Host Name: cdr2
      IP Address: 172.100.100.238
      Sub Netmask: 255.255.0.0
      Gateway:
      Name Server:
      Domain Name:

      Paging Space Information
      Total Paging Space: 8192MB
      Percent Used: 15%

      Volume Groups Information
      ==============================================================================
      rootvg:
      PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
      hdisk0 active 546 4 00..00..00..00..04
      hdisk1 active 546 0 00..00..00..00..00
      ==============================================================================

      cdr2vg:
      PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
      hdisk6 active 799 5 00..00..00..00..05
      hdisk7 active 799 0 00..00..00..00..00
      ==============================================================================

      INSTALLED RESOURCE LIST

      The following resources are installed on the machine.
      +/- = Added or deleted from Resource List.
      • = Diagnostic support not available.

      Model Architecture: chrp
      Model Implementation: Multiple Processor, PCI bus

      + sys0 System Object
      + sysplanar0 System Planar
      • vio0 Virtual I/O Bus
      • vsa1 U787F.001.DPM2A4M-P1-T2 LPAR Virtual Serial Adapter
      • vty1 U787F.001.DPM2A4M-P1-T2-L0 Asynchronous Terminal
      • vsa0 U787F.001.DPM2A4M-P1-T1 LPAR Virtual Serial Adapter
      • vty0 U787F.001.DPM2A4M-P1-T1-L0 Asynchronous Terminal
      • pci2 U787F.001.DPM2A4M-P1 PCI Bus
      • pci11 U787F.001.DPM2A4M-P1 PCI Bus
      + ent2 U787F.001.DPM2A4M-P1-C1-T1 10/100/1000 Base-TX PCI-X Adapter (14106902)
      • pci12 U787F.001.DPM2A4M-P1 PCI Bus
      • pci9 U787F.001.DPM2A4M-P1 PCI Bus
      + usbhc0 U787F.001.DPM2A4M-P1 USB Host Controller (33103500)
      + usbhc1 U787F.001.DPM2A4M-P1 USB Host Controller (33103500)
      • pci10 U787F.001.DPM2A4M-P1 PCI Bus
      + ent0 U787F.001.DPM2A4M-P1-T5 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
      + ent1 U787F.001.DPM2A4M-P1-T6 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
      • pci1 U787F.001.DPM2A4M-P1 PCI Bus
      • pci8 U787F.001.DPM2A4M-P1 PCI Bus
      + fcs0 U787F.001.DPM2A4M-P1-C6-T1 FC Adapter
      • fscsi0 U787F.001.DPM2A4M-P1-C6-T1 FC SCSI I/O Controller Protocol Device
      • dac0 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB 1722-600 (600) Disk Array Controller
      • dac1 U787F.001.DPM2A4M-P1-C6-T1-W200500A0B821D7CB 1722-600 (600) Disk Array Controller
      • fcnet0 U787F.001.DPM2A4M-P1-C6-T1 Fibre Channel Network Protocol Device
      • pci5 U787F.001.DPM2A4M-P1 PCI Bus
      + sa0 U787F.001.DPM2A4M-P1-C3-T1 2-Port Asynchronous EIA-232 PCI Adapter
      + tty0 U787F.001.DPM2A4M-P1-C3-T1-L0 Asynchronous Terminal
      • pci3 U787F.001.DPM2A4M-P1 PCI Bus
      • ide0 U787F.001.DPM2A4M-P1-T12 ATA/IDE Controller Device
      + cd0 U787F.001.DPM2A4M-P4-D2 IDE DVD-ROM Drive
      • pci4 U787F.001.DPM2A4M-P1 PCI Bus
      + sisscsia0 U787F.001.DPM2A4M-P1 PCI-X Dual Channel Ultra320 SCSI Adapter
      + scsi0 U787F.001.DPM2A4M-P1-T10 PCI-X Dual Channel Ultra320 SCSI Adapter bus
      + hdisk0 U787F.001.DPM2A4M-P1-T10-L5-L0 16 Bit LVD SCSI Disk Drive (73400 MB)
      + hdisk1 U787F.001.DPM2A4M-P1-T10-L8-L0 16 Bit LVD SCSI Disk Drive (73400 MB)
      + ses0 U787F.001.DPM2A4M-P1-T10-L15-L0 SCSI Enclosure Services Device
      + scsi1 U787F.001.DPM2A4M-P1-T11 PCI-X Dual Channel Ultra320 SCSI Adapter bus
      • pci6 U787F.001.DPM2A4M-P1 PCI Bus
      • pci7 U787F.001.DPM2A4M-P1-C5 PCI Bus
      + lai0 U787F.001.DPM2A4M-P1-C5-T1 GXT135P Graphics Adapter
      • pci0 U787F.001.DPM2A4M-P1 PCI Bus
      + L2cache0 L2 Cache
      + mem0 Memory
      + proc0 Processor
      + proc2 Processor
      + proc4 Processor
      + proc6 Processor
      + hdisk2 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB-L0 1722-600 (600) Disk Array Device
      + hdisk3 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB-L1000000000000 1722-600 (600) Disk Array Device
      + hdisk4 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB-L2000000000000 1722-600 (600) Disk Array Device
      + hdisk5 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB-L3000000000000 1722-600 (600) Disk Array Device
      + hdisk6 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB-L4000000000000 1722-600 (600) Disk Array Device
      + hdisk7 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB-L5000000000000 1722-600 (600) Disk Array Device
      + hdisk8 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB-L6000000000000 1722-600 (600) Disk Array Device
      + hdisk9 U787F.001.DPM2A4M-P1-C6-T1-W200400A0B821D7CB-L7000000000000 1722-600 (600) Disk Array Device

      I have try another some version of nmon get from
      http://www-941.haw.ibm.com/collaboration/wiki/display/WikiPtype/nmon
      just like nmon4aix_11e.tar.gz (nmon_aix53),it has the same problem;

      thanks steven
      • nagger
        nagger
        1594 Posts
        ACCEPTED ANSWER

        Re: A nmon problem wtih errno=13

        ‏2008-04-28T11:08:50Z  in response to SystemAdmin
        OK thanks for the output nothing bad I can see here.

        There must be something subtly difference in the environment.
        Any reason you are using bash?

        Next questions:
        1) As both the root and the regular user can we have the output from: echo $PATH
        Perhaps the regular user is restricted or can't find a program.

        2) Can you try a Korn shell user?
        I don't even have bash on my system so that is hard to test.

        3) Next, something is failing lets be very certain about what. We can do that with truss
        run truss nmon_aix53
        Can you give us the last ~30 lines of output.

        4) If this does turn out to be perfstat_diskadapter() then we have a libperfstat bug (again).
        Can you return the file /usr/include/libperfstat.h ?
        There should not be any differences from the version used to compile nmon in the first place

        5) nmon version 12 has some work arounds in this area. Please download nmon12 and give that ago.

        Hopefully, we will find a fix in the end, thanks N
        • SystemAdmin
          SystemAdmin
          2404 Posts
          ACCEPTED ANSWER

          Re: A nmon problem wtih errno=13

          ‏2008-04-29T10:06:22Z  in response to nagger
          1)regular user
          $ echo $PATH
          /usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java14/jre/bin:/usr/java14/bin
          root
          1. echo $PATH
          /usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java14/jre/bin:/usr/java14/bin
          2)I used ksh to do all

          3)please check the attach files

          4)please check the attach files too

          5)I will find the version 12 and try it again,Can you tell me where to down it?

          thanks a lot ,steven
          • SystemAdmin
            SystemAdmin
            2404 Posts
            ACCEPTED ANSWER

            Re: A nmon problem wtih errno=13

            ‏2008-04-29T11:34:16Z  in response to SystemAdmin
            I have download nmon12 from
            http://www-941.haw.ibm.com/collaboration/wiki/download/attachments/437/nmon4aix12e.tar.gz?version=1

            I got the same error!

            AIX version 5.3.0.0 and starting up nmon nmon12e_aix534
            ./nmon12e_aix534
            ERROR: Assert Failure in file="nmon12.c" in function="main" at line=4442
            ERROR: Reason=System call returned -1
            ERROR: Expression=
            ERROR: errno=13
            ERROR: errno means : Permission denied
            ERROR: Sizeof cpu=448 cpu_total=528 disk=472 diskadapter=192 diskpath=304 disktotal=64, memory=176 netbuff=120 netif=208 netiftotal=64 paging=240 partition=232 protocol=432
            • nagger
              nagger
              1594 Posts
              ACCEPTED ANSWER

              Re: A nmon problem wtih errno=13

              ‏2008-05-01T09:01:12Z  in response to SystemAdmin
              Thanks for the information

              First please delete nmon11d from your systems and only use nmon12 from now on.
              Especially, as nmon11d was replaced with nmon11e in July 2006!

              The failure is happening in the libperfstat library call and this library works out the details by getting the disk to disk adapter map from the ODM. The root user can open this read/write but when the regular user does this it fails, so it opens it then opens it read only. After dozens of these it fails with what looks like a signal.

              Have you got ulimits set very low for users?
              The nmon program will be growing in size as this data is extracted and may have hit a size limit.

              Have you been meddling with the ODM or fixing ODM issues?
              This usually involves fixing disk issues which is in the right area.
              Perhaps your ODM is not root only use - no idea on what to do about that.

              We had AIX bugs in AIX 5.3 ML3 that looks much like this but happened for all users as far as I can remember. nmon12 attempts to work around these library failures but your system then fails the second attempt. The library call to determine the number of adapters works but fetching the stats for each adapter fails.
              Can you try setting the NMON_IGNORE_ASSERT shell variable and running nmon12?
              In Korn shell:
              export NMON_IGNORE_ASSERT=1
              nmon12e_aix534

              This tells nmon12 to not halt when it hits ASSERT problems and it will decide you have zero disk adapter but at least carry on.

              Good luck and let us know it that at least gets nmon running (even without adapter stats), N

              ps: AIX 5.3 TL4 is very old now, I suspect an update is in order to TL7 but there is a concluding Service Pack for 5.3 TL4 that you don't appear to have taken, yet. Yes I know, if its not borken ... don't fix it.
              • SystemAdmin
                SystemAdmin
                2404 Posts
                ACCEPTED ANSWER

                Re: A nmon problem wtih errno=13

                ‏2008-05-05T01:31:49Z  in response to nagger
                hi,nagger
                [1]
                regular user
                $ ulimit -a
                time(seconds) unlimited
                file(blocks) unlimited
                data(kbytes) unlimited
                stack(kbytes) 4194304
                memory(kbytes) 32768
                coredump(blocks) 2097151
                nofiles(descriptors) 2000

                root
                1. ulimit -a
                time(seconds) unlimited
                file(blocks) unlimited
                data(kbytes) unlimited
                stack(kbytes) 4194304
                memory(kbytes) 32768
                coredump(blocks) 2097151
                nofiles(descriptors) 2000

                It seem the same between regular user and root

                2,when I use nmon12e_aix534 and set export NMON_IGNORE_ASSERT=1

                both in Korn shell and bash shell it works well.

                thanks a lot!

                Message was edited by: Steven@Tulip
                Updated on 2008-05-05T01:31:49Z at 2008-05-05T01:31:49Z by SystemAdmin
                • SystemAdmin
                  SystemAdmin
                  2404 Posts
                  ACCEPTED ANSWER

                  Re: A nmon problem wtih errno=13

                  ‏2008-05-05T02:22:36Z  in response to SystemAdmin
                  hi ,nagger

                  when I type "nmon" it can run ok,but when I use this command

                  it also have some error

                  -bash-3.00$ nmon -f -s 3 -c 10
                  ERROR: unexpected result from AIX libperfstat library system call.
                  ERROR: perfstat_diskadapter(first-adapter="", buffer=0x20348548, size=192, adapters=3)
                  ERROR: Working around this error by setting adapter count to zero
                  -bash-3.00$ ksh
                  $ nmon -f -s 3 -c 10
                  ERROR: unexpected result from AIX libperfstat library system call.
                  ERROR: perfstat_diskadapter(first-adapter="", buffer=0x20348548, size=192, adapters=3)
                  ERROR: Working around this error by setting adapter count to zero

                  thanks steven

                  Message was edited by: Steven@Tulip

                  Message was edited by: Steven@Tulip
                  Updated on 2008-05-05T02:22:36Z at 2008-05-05T02:22:36Z by SystemAdmin
                  • nagger
                    nagger
                    1594 Posts
                    ACCEPTED ANSWER

                    Re: A nmon problem wtih errno=13

                    ‏2008-05-06T07:37:58Z  in response to SystemAdmin
                    But did you check if nmon actually carried on afterwards with: ps -ef | grep nmon
                    I think nmon reports the error and carries on running but then drops into the back ground so that you terminal session control is returned back o you.

                    ta N
                    • SystemAdmin
                      SystemAdmin
                      2404 Posts
                      ACCEPTED ANSWER

                      Re: A nmon problem wtih errno=13

                      ‏2008-05-06T08:13:19Z  in response to nagger
                      yes! I can see it running on the back ground !

                      Is there any problem with this error or I can ignore this error?

                      thanks a lot !

                      steven
                      • nagger
                        nagger
                        1594 Posts
                        ACCEPTED ANSWER

                        Re: A nmon problem wtih errno=13

                        ‏2008-05-06T15:59:37Z  in response to SystemAdmin
                        You can ignore the message it is only telling you that there is no adapter stats but will run anyway.

                        thanks Nigel
                        • SystemAdmin
                          SystemAdmin
                          2404 Posts
                          ACCEPTED ANSWER

                          Re: A nmon problem wtih errno=13

                          ‏2008-05-07T02:17:50Z  in response to nagger
                          OK,thanks a lot!

                          steven
                          • varadh_007
                            varadh_007
                            1 Post
                            ACCEPTED ANSWER

                            Re: A nmon problem wtih errno=13

                            ‏2012-01-23T11:20:13Z  in response to SystemAdmin
                            Hi Guys,

                            Once i set this variable "export NMON_IGNORE_ASSERT=1" and if i start the nmon with normal user it is generating core file and it get stopped within 12 hours . Also generating same error

                            "ERROR,T0004, Assert Failure in file="nmon12.c" in function="main" at line=8224
                            ERROR,T0004, Reason=NULL pointer
                            ERROR,T0004, Expression=[p->procs = MALLOC(sizeof(struct procentry64 ) * n )]
                            ERROR,0004, errno=12
                            ERROR,T0004, Sizeof cpu=512 cpu_total=592 disk=472 diskadapter=192 diskpath=304 disktotal=64, memory=176 netbuff=120 netif=20
                            8 netiftotal=64 paging=240 partition=384 protocol=720

                            Kindly check and revert back .