PDB files are generated if a project is built by using the /Zi or /ZI (Produce PDB Information) compiler switch... Generating PDB files for release executables does not affect any optimizations, or significantly alter the size of the generated files... For this reason, you should always produce PDB files...
We added /Zi to the compiler (cl.exe) and it produced a PDB file; however, when we grabbed a dump of the process and loaded it in windbg, it said that it couldn't find the PDB file:
0:000> ld ibmaiodbg *** WARNING: Unable to verify checksum for ibmaiodbg.dll *** ERROR: Symbol file could not be found. Defaulted to export symbols for ibmaiodbg.dll - Symbols loaded for ibmaiodbg
This was strange because the pdb file definitely existed in .sympath and we called .reload. We used the following command to get detailed loading information:
0:000> !sym noisy; .reload /f noisy mode - symbol prompts on DBGHELP: No debug info for ibmaiodbg.dll. Searching for dbg file ... DBGHELP: c:\program files\ibm\websphere7\appserver\bin\ibmaiodbg.pdb - E_PDB_CORRUPT
Strange! We tried basic manipulation of the PDB and everything returned E_PDB_CORRUPT. For example:
C:\Program Files\IBM\WebSphere7\AppServer\bin>"C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\pdbcopy.exe" ibmaiodbg.pdb test.pdb -p Can't open pdb file ibmaiodbg.pdb Error: EC_CORRUPT
Next we found the symchk tool which gave a hint:
C:\Program Files\IBM\WebSphere7\AppServer\bin>"C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\symchk.exe" /v ibmaiodbg.dll /s . ... SYMCHK: ibmaiodbg.dll FAILED - Built without debugging information. SYMCHK: FAILED files = 1
Also strange because we definitely compiled with /Zi:
cl /Fdibmaiodbg.pdb /Zi ...
The /Zi flag implies /DEBUG (not to be confused with a debug build), and although LINK doesn't have a /Zi flag, we noticed that it also has a /DEBUG flag. The key line was this:
It updates the PDB during subsequent builds of the program.
What happened was that the compile and link steps happened independently, so although we passed /Zi to compile (which implied /DEBUG), we needed to pass /DEBUG explicitly to link. What's confusing is that the compile step produced a PDB, but then the link step with /DEBUG will update that same PDB file and add more symbols into it. With this change, running symchk worked:
"C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\symchk.exe" /v ibmaiodbg.dll /s . DBGHELP: ibmaiodbg - private symbols & lines ... SYMCHK: PASSED + IGNORED files = 1
Update (June 3rd): It turns out that the E_PDB_CORRUPT error had nothing to do with the missing /DEBUG parameter to LINK, but instead they actually were corrupted by our build process. However, it was serendipitous that we found the /DEBUG parameter to LINK as that is absolutely necessary for full PDB symbol files. We also found that with a "release" DLL, windbg will not be able to show parameters and line numbers even with private symbols. This is okay since you normally just want valid stack traces for production builds for native memory issues and crashes. Finally, !chksym MODULE is a useful windbg command to see if the DLL and PDB have matching signatures.