Question & Answer
Question
I have used the Reliability procedure in SPSS Statistics to report the mixed model intraclass correlations for each of two groups. Three raters rated images from each of 20 patients, for example, from group 1. The same three raters rated images for a different set of patients from group 2. All patients were rated by all 3 raters so raters is a fixed factor. I wish to test whether the Intraclass Correlation (ICC) from group 1 is equal to the ICC from group 2. Can this test be performed in Statistics?
Answer
|
SPSS Statistics does not have an option within the Reliability procedure or other procedures to test group difference between Cronbach alphas or ICCs for a set of items. (The Average Measures ICC for the 2-way mixed model is equal to Cronbach's alpha. ) An F test is formed from the ratio of ( 1 - smaller-alpha) over (1 - larger-alpha) and compared to an F distribution with numerator and denominator degrees of freedom equal to (N for larger alpha - 1) and (N for smaller alpha - 1), respectively. The order of the Ns for degrees of freedom may seem to be reversed, but this is clearly the arrangement in Feldt et al (1987). The syntax command set below generates case-level data for items on a ten-item scale (ITEMA1 to ITEMA10) for cases in each of two groups (GROUP = 1 or 2). The SPLIT FILE command is run with GROUP designated as the Split File variable. The Reliability procedure is run with /MODEL=ALPHA. An OMS (Output Management System) command preceding RELIABILITY directs the "Reliability Statistics" output to an SPSS data (.sav) file. With the original data split by the Group variable, the OMS command in the example saves the statistics for the separate groups as separate sets of variables in the same row of the new data file, rather than as a single set of variables with a row for each group. The resulting structure simplifies the Compute commands for computation of the F value and its significance. The statistical test also requires the listwise-complete sample size for each group. (Reliability omits any cases that are missing on any of the variables in the /VARIABLES list.) Unfortunately, the listwise N is printed in the Case Processing Summary table of Reliability output, which cannot be saved in the same OMS-generated file as the 'Reliability Statistics' output because of size and format differences between the two tables. (You would get a warning that the "stub sizes" don't agree,) You could precede the Reliability command with a separate OMS command to direct the Case Processing output to a separate file. An alternate strategy that is used in the example below is to use the Aggregate procedure to save the listwise N to a single case file to be merged with the file saved by OMS. The computation of F and it's significance can then be performed with COMPUTE commands on the merged file, as demonstrated in the example syntax commands below. The generated case-level data in the example below has some missing values spread among the item responses, as might occur with real data. To get the listwise N for each of the respective groups , the variables FULLITEMS1 and FULLITEMS2 were computed in the case-level file. The FULLITEMS variable that corresponds to a case's group equals 1 if the case had full data on items in the scales; 0, otherwise. The Aggregate procedure was used to save the listwise Ns as the sums across cases of FULLITEMS1 and FULLITEMS2. Several file paths are included among the commands. You would replace the folder and drive names with those on your own computer. The data with the test item responses is saved as randat_10items_2groups.sav and then read with a GET FILE command. (The GET FILE command is commented out below with an asterisk ('*') in column 1.) That GET FILE command (with your own path and file names substituted) is where your commands would generally begin, reading an existing file with real data. The remaining files named in the commands are generated by the command set, so you could use those file names if you wished. You will need to replace the variable list in the RELIABILITY command with the variable names that represent the scale items in your own data set. The variable list in the RELIABILITY command below is 'itema1 to itema10'. If the scale variables in your file are not all contiguous in the data set, with no unwanted variables among them, you will need to list the individual variable names, rather than using the '<first> TO <last>' convention. @1_CronbachsAlpha is the name that OMS used to save the alpha for group1 and @2_CronbachsAlpha is the name that OMS used to save the alpha for group 2. You can set up an OMS command from the Utilities->OMS Control Panel menu. When building an OMS application, you generally have to actually run an OMS-RELIABILITY (or other statistical procedure) command set and then open the resulting file to be sure of the variable names and structures that will appear in that file. ############ Example of SPSS Syntax commands to implement the Feldt (p 96) test for 2 independent alphas #############. * generate responses on 10-item test for each of two samples of 60 people and then * test whether the Cronbach alphas differ. * Change file paths to reflect your drive and folder names . * Based on p. 96 of : * Feldt, L.S., Woodruff, D.J., & Salih, F.A. (1987). Statistical inference for coefficient alpha. * Applied Psychological Measurement, 11(1), 93-103. set seed = 200000 . input program . loop id = 1 to 120. compute group = (id > 60) + 1. compute pers = normal(2). do repeat ia = itema1 to itema10 . compute ia = 5+ rnd((pers + normal(2 + (group-1)))/(2+(group-1))). end repeat. end case. end loop. end file. end input program. execute. format group (f4). * generate some missing values. do repeat im = itema1 to itema10 . if (uniform(1) < .01) im = $sysmis. end repeat. execute. SAVE OUTFILE='C:\spssjobs\Compare_Calpha\randat_10items_2groups.sav' /COMPRESSED. * GET FILE ='C:\spssjobs\Compare_Calpha\randat_10items_2groups.sav' . * given an existing data set with variables group and itema1 to itema10, * your analysis would begin here . if (group = 1) fullitems1 = (nmiss(itema1 to itema10) = 0) . if (group = 2) fullitems2 = (nmiss(itema1 to itema10) = 0) . compute brk = 1. execute. AGGREGATE /OUTFILE='C:\spssjobs\Compare_Calpha\TestA_fullitemN.sav' /BREAK=brk /Ng1 Ng2 = SUM(fullitems1 fullitems2). split file by group. OMS /SELECT TABLES /IF COMMANDS = ["Reliability"] SUBTYPES = ["Reliability Statistics"] /DESTINATION FORMAT = SAV NUMBERED = TableNumber_ OUTFILE = "C:\spssjobs\Compare_Calpha\TestA_RelSummary.sav" /COLUMNS SEQUENCE = [RALL CALL LALL] /tag = 'TestA_2gp'. RELIABILITY /VARIABLES=itema1 to itema10 /SCALE('ALL VARIABLES') ALL /MODEL=ALPHA /STATISTICS=DESCRIPTIVE SCALE /SUMMARY=TOTAL MEANS VARIANCE /ICC=MODEL(MIXED) TYPE(CONSISTENCY) CIN=95 TESTVAL=0. omsend tag = 'TestA_2gp'. get file = "C:\spssjobs\Compare_Calpha\TestA_RelSummary.sav" MATCH FILES /FILE=* /FILE='C:\spssjobs\Compare_Calpha\TestA_fullitemN.sav'. EXECUTE. * Find larger and smaller alpha for denominator and numerator of F, respectively. do if (@2_CronbachsAlpha >= @1_CronbachsAlpha). compute alpha_num = @1_CronbachsAlpha. compute alpha_den = @2_CronbachsAlpha. compute df_num = NG2 - 1. compute df_den = NG1 - 1. ELSE. compute alpha_num = @2_CronbachsAlpha. compute df_num = NG2 - 1. compute alpha_den = @1_CronbachsAlpha. compute df_num = NG1 - 1. compute df_den = NG2 - 1. end if. compute F_Calpha_dif = (1 - alpha_num)/(1 - alpha_den) . COMPUTE sig_FCalpha = SIG.F(F_Calpha_dif,df_num, df_den) . formats sig_FCalpha (f8.4). list @1_CronbachsAlpha Ng1 @2_CronbachsAlpha Ng2 F_Calpha_dif sig_FCalpha . SAVE OUTFILE='C:\spssjobs\Compare_Calpha\TestA_2group_wFsig.sav' /COMPRESSED. |
Related Information
[{"Product":{"code":"SSLVMB","label":"IBM SPSS Statistics"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"--","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"Not Applicable","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Was this topic helpful?
Document Information
Modified date:
16 April 2020
UID
swg21498525