With 9.5.1, RDi now installs and runs on Mac OS X 10.11.  Most of the functions are the same as in Windows/Linux but there are differences.  http://www-01.ibm.com/support/docview.wss?uid=swg27038481#9510 has an overview of the limitations.

In terms of installing, you can use the web install method (install IBM Installation Manger for OS X and then point to the IBM servers for RDi 951 - see http://www-01.ibm.com/support/docview.wss?uid=swg21667734 for details) or install from a downloaded image of 9.5.1 from Passport Advantage or ESS.  (A third option is to download the 9.5.1 update zip from FixCentral and specify that along with the 9.5 downloaded image in IBM Installation Manager.)  

If installing from a downloaded image, there can be issues launching the installer or launchpad.  See http://www.ibm.com/support/knowledgecenter/SSAE4W_9.5.1/com.ibm.etools.iseries.install.doc/topics/c_install_scenarios.html (if not using the launchpad) and http://www.ibm.com/support/knowledgecenter/SSAE4W_9.5.1/com.ibm.etools.iseries.install.doc/topics/t_start_install_launchpad.html for some OS X specific workarounds if you run into issues.

Another item to note is that the integration with Access Client Solutions (e.g. Launch Run SQL Scripts, Launch Printer Output, Launch 5250 Emulator actions) requires Legacy Java 6 to also be installed in order to work.

Even with the limitations, RDi on Mac provides a rich set of tools for IBM i development.