Topic
  • 5 replies
  • Latest Post - ‏2012-11-21T21:23:58Z by SystemAdmin
SystemAdmin
SystemAdmin
456 Posts

Pinned topic Significance Testing with Weighting

‏2012-11-21T18:36:36Z |
Hello,

Is there a correction in SPSS to significance testing means that are weighted?

I have seen quite a bit about it and solutions using other software (http://www.spsstools.net/Tutorials/WEIGHTING.pdf or http://www.analyticalgroup.com/download/Alternative%20Approaches.pdf), but I have not seen solutions that internally correct for the base inflation.

The weights for this particular analysis cannot be sampled balanced and could range from 0 to anything, really.

Thanks.
  • SystemAdmin
    SystemAdmin
    456 Posts

    Re: Significance Testing with Weighting

    ‏2012-11-21T18:59:44Z  
    The treatment of weights in Statistics varies somewhat, but in general, significance tests consider the N to be the sum of the weights except for the Complex Samples procedures, which provide full support for sampling schemes. Using the other procedures, you should at least normalize the weights so that they sum to the actual N.
  • SystemAdmin
    SystemAdmin
    456 Posts

    Re: Significance Testing with Weighting

    ‏2012-11-21T20:17:35Z  
    The treatment of weights in Statistics varies somewhat, but in general, significance tests consider the N to be the sum of the weights except for the Complex Samples procedures, which provide full support for sampling schemes. Using the other procedures, you should at least normalize the weights so that they sum to the actual N.
    Hi,

    Can you elaborate? I may be confused on the terminology, but I do not think I can normalize the weights as the weights are assigned on each analysis. That is: I am comparing the mean of one variable for respondents in one wave against the mean for the other wave, and each respondent's mean is weighted by a specific number for that respondent, which could vary from 0 to anything. This happens for a lot of analyses.

    I am not applying a weight to a group of respondents to bring them in line with the universe of respondents, in which case I would sample balance them.

    Thanks
  • SystemAdmin
    SystemAdmin
    456 Posts

    Re: Significance Testing with Weighting

    ‏2012-11-21T20:24:09Z  
    Hi,

    Can you elaborate? I may be confused on the terminology, but I do not think I can normalize the weights as the weights are assigned on each analysis. That is: I am comparing the mean of one variable for respondents in one wave against the mean for the other wave, and each respondent's mean is weighted by a specific number for that respondent, which could vary from 0 to anything. This happens for a lot of analyses.

    I am not applying a weight to a group of respondents to bring them in line with the universe of respondents, in which case I would sample balance them.

    Thanks
    I don't understand what these weights are, so I can't offer any advice.
  • SystemAdmin
    SystemAdmin
    456 Posts

    Re: Significance Testing with Weighting

    ‏2012-11-21T21:10:10Z  
    I don't understand what these weights are, so I can't offer any advice.
    Thanks Jon, I am trying to be a little vague to blind it, but let me try.

    Weight= number of vegetables you eat. You eat 10, I eat 1. Call this veg_num.

    Then, out of the weight, you eat 5 cucumbers. I eat 1 cucumber of my 1 vegetables.

    When I want to find the mean percent for cucumbers, I want to weight your 50% (cuke_pct) more than my 100% (cuke_pct), so I weight it by the 10 and 1 from veg_num.

    Last time we surveyed, you ate 10 veg_num and I had 5 veg_num, and it was 1 cucumber for you out of 10 and 5 cuke for me. So it was 10% cuke_wt for you and 100% cuke_wt for me. I want to weight your 10% by your vegetable volume and my 5% by mine.

    Then I want to compare the two waves of us, me and you, from last time and this time, on our mean percent of cucumbers.

    weight by veg_num.
    T-TEST GROUPS=wave(1 2)
    /MISSING=ANALYSIS
    /VARIABLES=cuke_pct
    /CRITERIA=CI(.95).
    weight off.
  • SystemAdmin
    SystemAdmin
    456 Posts

    Re: Significance Testing with Weighting

    ‏2012-11-21T21:23:58Z  
    Thanks Jon, I am trying to be a little vague to blind it, but let me try.

    Weight= number of vegetables you eat. You eat 10, I eat 1. Call this veg_num.

    Then, out of the weight, you eat 5 cucumbers. I eat 1 cucumber of my 1 vegetables.

    When I want to find the mean percent for cucumbers, I want to weight your 50% (cuke_pct) more than my 100% (cuke_pct), so I weight it by the 10 and 1 from veg_num.

    Last time we surveyed, you ate 10 veg_num and I had 5 veg_num, and it was 1 cucumber for you out of 10 and 5 cuke for me. So it was 10% cuke_wt for you and 100% cuke_wt for me. I want to weight your 10% by your vegetable volume and my 5% by mine.

    Then I want to compare the two waves of us, me and you, from last time and this time, on our mean percent of cucumbers.

    weight by veg_num.
    T-TEST GROUPS=wave(1 2)
    /MISSING=ANALYSIS
    /VARIABLES=cuke_pct
    /CRITERIA=CI(.95).
    weight off.
    Ok, so these are not statistical weights at all. They don't represent sampling or replication. They are importance weights and have no bearing on t test degrees of freedom.
    BTW, you might find it interesting to use the Trees procedure with an influence variable. But don't try to use this weight in the usual SPSS sense. You might also want to compute a weighted mean and then apply a t test to the aggregates using the correct number of degrees of freedom.