SELECTION OF A STATISTICAL ANALYSIS APPROACH Even at the data collection stage,

SELECTION OF A STATISTICAL ANALYSIS APPROACH
Even at the data collection stage, the social work researcher needs to know what type of data analysis will facilitate an answer to the research question. The researcher should understand the purpose of each method of analysis, the characteristics that must be present in the study for the design to be appropriate, and any weaknesses of the design that might limit the usefulness of the results. Only then can the researcher select the appropriate design.
Choosing the appropriate design enables the social work researcher to gather the most relevant information about the relationship being studied. Notice that it is not the statistical test itself that deems the research valid; rather, it is the research design. Social workers must be aware of and adjust any limitations of their chosen design that may impact the validity of the study.
In this Discussion, you examine a case study involving a quantitative design, determining whether the statistical information supports the program’s efficacy and whether there are limiting factors.
RESOURCES
Dudley, J. R. (2020). Social work evaluation: Enhancing what we do (3rd ed.). Oxford University Press.Chapter 10, “Analyzing Evaluation Data” (pp. 255–275)
Flannelly, K. J., Flannelly, L. T., & Jankowski, K. R. B. (2018). Threats to the internal validity of experimental and quasi-experimental research in healthcareLinks to an external site.. Journal of Health Care Chaplaincy, 24(3), 107–130. https://doi.org/10.1080/08854726.2017.1421019
Document: A Short Course in Statistics Download A Short Course in Statistics(PDF)
Required Media
Walden University, LLC. (2022). Social work case studiesLinks to an external site. [Interactive media]. https://waldenu.instructure.comNavigate to the Chi-Square case study.
TO PREPARE
Review the Learning Resources on analyzing evaluation data, threats to internal validity, and statistics.
Access the Social Work Case Studies media and navigate to the Chi-Square case study.
As you review the case, consider the confounding variables—that is, factors that might explain the difference between those in the program and those waiting to enter the program.
BY DAY 3
Post a brief outline of the case study and consider the conclusion that “the vocational rehabilitation intervention program may be effective at promoting full-time employment.”
What statistical information shows whether the program was effective (or not)?
Review the factors that limit the internal validity of a study (history, maturation, testing, instrumentation, statistical regression, selection bias, and attrition).
Select and explain which of these factors might limit the ability to draw conclusions regarding cause-and-effect relationships.
BY DAY 6
Respond to at least two colleagues by explaining how that colleague might rule out one of the confounding variables that they identified.
Response 1 to SJR
This study was conducted to determine if a recently instituted vocational rehabilitation program for newly paroled prison inmates was effective. The study determined that the program might be effective at promoting full-time employment among recent paroled inmates. The program was evaluated using a Chi-Square. The statistical information demonstrated that, of the paroled inmates in the intervention group, 60% (or 18 of the 30 participants) obtained full time employment. Conversely, a little over 20% (or 6 of the 29 participants) of the comparison group obtained full-time employment.
There are several factors that can limit internal validity of a study of this kind. History is one of those factors. History refers to events that occur in the environment that can change the outcomes in a study (Flannelly et al., 2018). It is described by Flannelly et al. (2018) as the experience of external events. Maturation is another factor. This encompasses changes such as those related to age, being fatigued, or having some type of disease or illness (Flannelly et al., 2018). Another factor is the testing itself. The study needs to show that there is a clear and definite relationship between the test and what it is measuring (Flannelly et al., 2018). Flannelly et al. (2018) noted that there can be reactivity associated with a study that would affect the outcomes. Reactivity refers to the way in which a test subject modifies their behavior due to the fact that they are being studied (Jimenez-Buedo, 2019). Instrumentation is yet another factor that can affect internal validity. This refers to the concept that “any change in measurement can pose a threat to internal validity.” (Flannelly et al., 2018). The term instrument can refer, in this context, to an electronic or mechanical instrument or to human researchers and how they judge or rate a dependent variable (Flannelly et al., 2018). Statistical regression can also cause a natural variation in data to appear as real change when in fact, it is not (Barnett et al., 2005). Another factor is selection bias. This refers to the possibility of bias when selecting the participants for the study (Flannelly et al., 2018). If selection bias occurs, there may be important differences in the study’s intervention group and those in the control group (Flannelly et al., 2018). These differences could affect the outcomes of the study as the groups would not then be equivalent (Flannelly et al., 2018).
This researcher views factors that influence the internal validity in this study as possibly maturation and history. Flannelly et al. (2018) discusses that maturation involves bodily changes and biological changes. In regard to this study, if some of the participants in either group were ill or had some type of diagnosed disease, that could affect their ability to obtain full-time employment. Additionally, age is not mentioned for participants in either group. Age can play an important role in obtaining jobs. The other chosen possible influence, history, is comprised of everyday experiences of the participants that could affect the study outcome (Flannelly et al., 2018). This could even be that some of the participants may be able to communicate better than others and therefore be able to make a better impression on a possible employer, thus enhancing their potential in obtaining a job. There is also no mention of education about any of the participants. Some of the participants may have a higher education than others or have more prior experience in some form of job-related field that would make it easier for them to obtain a job.
Resources
Barnett, A. G. (2005). Regression to the Mean: What It Is and How to Deal With It. International Journal of Epidemiology, 215 – 220.
Flannelly, K. J. (2018). Threats to the Internal Validity of Experimental and Quasi-Experimental Research in Healthcare. Journal of Health Care Chaplaincy, 107 – 130.
Jimenez-Buedo, M. (2021). Reactivity in Social Scientific Experiments: What Is It and How Is It Different (and Worse) Than a Placebo Effect? European Journal for Philosophy of Science, 1 – 22.
Walden University, LLC. (2022). Social Work Case Studies [Interactive media]. https://waldenu.instructure.com
Response 2 to AG
Post a brief outline of the case study and consider the conclusion that “the vocational rehabilitation intervention program may be effective at promoting full-time employment.”
The purpose of this case study is to investigate the efficiency of a vocational rehabilitation intervention program in terms of increasing participants likelihood of obtaining full-time job after their release from prison. The research employs a technique known as a quasi-experimental comparison between a group of participants who went through the program known as the intervention group and a waiting list of people who have not yet participated in the program known as the comparison group. Participants in the intervention group were more likely to have full-time jobs than those in the comparison group. Nevertheless, the study has a number of drawbacks, the most notable of which are the absence of a random assignment and the potential for biased selection (Walden University, 2022).
What statistical information shows whether the program was effective (or not)?
The chi-square test for independence, which was used to determine whether or not the program was successful, found that the difference between the intervention group and the comparison group was highly significant, with a p value of.003 (Walden University, 2022).This information was used to determine whether or not the program was successful. However, this is a significant difference between the employment outcomes of the two groups, with the intervention group having a greater likelihood of being employed full-time as compared to the comparison group, but the study suffers from a number of shortcomings that make it challenging to draw any conclusions about the relationship between cause and effect.
Review the factors that limit the internal validity of a study (history, maturation, testing, instrumentation, statistical regression, selection bias, and attrition).
The history, maturation, testing, instrumentation, statistical regression, selection bias, and attrition of the participants are the elements that limit the internal validity of an investigation.
The History, is the important events that took place before the beginning of the investigation and could have an impact on the findings.
Maturation, regardless of the intervention or treatment, study participants may develop organically over time. This study’s outcome may alter as a result of this maturation, which is not always due to the intervention.
Testing, is the phenomenon in which individuals are more likely to alter their conduct when they are aware that they are being observed.
Instrumentation, it alludes to the possibility that the methods and tools utilized to collect data will evolve over the course of the research project.
Statistical Regression, is the observation that individuals who are on opposite ends of the scale for the independent variable are more likely to shift positions.
Selection bias, this occurs when study participants are not assigned at random, resulting in differences between participants in the intervention and control groups. This can make it challenging to solely credit any observed variations in outcome to the intervention.
Attrition, occurs when participants leave a study before it is finished. This may result in a non-representative sample and potential biases or confounds in the study’s outcome.
These concepts are said to be threats to the internal validity of experiments because they pose alternate explanations for the apparent causal relationship between the independent variable and dependent variable of an experiment if they are not adequately controlled (Flannelly, K. J et. al, 2018).
Select and explain which of these factors might limit the ability to draw conclusions regarding cause-and-effect relationships.
The ability to form conclusions about cause-and-effect relationships can be limited by a number of the criteria stated. One factor limit I chose is Attrition. Attrition, is absence of a random assignment process, it is impossible to determine with certainty whether the difference in job outcomes is the result of the program or whether it is due to other factors. Another factor limit is Selection bias. According to Flannelly, K. J. et al, (2018), it states study selection bias, study displays was the first group of participates in the program which increase their motivation to gain employment, while the second group is on the waitlist and may lack motivation. These factors limit validity because they offer an alternative explanation for the effects of the experiment. Due to this, there is not enough data to support the employment being a direct cause of the vocational program
References:
Flannelly, K. J., Flannelly, L. T., & Jankowski, K. R. B., (2018). Threats to the internal validity of experimental and quasi-experimental research in healthcare. Journal of Health Care Chaplaincy, 24(3), 107–130. https://doi.org/10.1080/08854726.2017.1421019
Walden University, LLC., (2022). Social work case studies. https://waldenu.instructure.com. Chi-Square case study.