Competency 5: Assessing Student Learning

Competency Description

Design or participate in teaching research or inquiry for purposes of improving student learning.

This competency requires designing and implementing systematic inquiry into teaching effectiveness through evidence-based assessment. I completed a mentored teaching project examining how collaborative marketing case study learning affects student outcomes across multiple dimensions: marketing knowledge development, analytical confidence, collaborative skill acquisition, and learning attitudes.

Teaching Research Project: Collaborative Learning in Marketing Case Studies

Research Context

Course: MKT 327 - Marketing Management

Term: Summer 2025 (July 1 - August 15, 6-week intensive)

Sample: 50 students who completed both pre-course (Week 1) and mid-course (Week 4) surveys Design: Matched-student repeated measures with comprehensive outcome assessment

Research Question and Expectations

Primary Research Question

“How does participation in collaborative marketing case studies affect students’ self-reported marketing concept familiarity, analytical confidence, and collaborative skills development over a 6-week intensive course?”

What I Expected to Find

By Week 1, I had already formed expectations based on my observations and prior research on collaborative learning:

Expected Outcome 1: Improved Self-Reported Familiarity
I expected students would report significant improvement in their marketing concept familiarity from Week 1 to Week 4. The initial survey responses showed most students felt not very familiar with marketing concepts, leaving substantial room for growth through collaborative case analysis.

Expected Outcome 2: Contribution Pattern Adjustment
I anticipated students would initially overestimate their contributions relative to groupmates—a common pattern in group work. After actual collaborative experience, I expected they’d adjust toward more realistic self-assessments of contributing “about equally.”

Expected Outcome 3: Benefits and Challenges Coexisting
I expected students would identify both logistical challenges (scheduling, communication) and learning benefits (diverse perspectives, knowledge sharing, error detection).

Expected Outcome 4: Widespread Confidence Growth
I predicted most students would report increased confidence in analyzing marketing cases after collaborative work.

Pedagogical Context

Collaborative Learning Structure:

14 groups of 5 students each (random assignment)
4 group case studies (Nike/Mayo/Louis Vuitton, Chase/Apple/Uber, Redbull/Bestbuy/Airbnb, IKEA/Starbucks/Honest Tea)
Group work comprised $40 \%$ of final grade
3 individual exams comprised $60 \%$ of final grade

Assessment Timeline:

Week 1: Pre-course survey (baseline)
Weeks 1-6: Four group case study assignments
Week 4: Mid-course survey (progress assessment)
Weeks 3, 5, 7: Three individual exams

Research Methodology

Data Collection Instruments

1. Pre-Course Survey (Week 1) - Baseline Assessment

Marketing familiarity: 5 -point Likert scale ( $1=$ not familiar, $5=$ extremely familiar)
Expected contribution patterns: 5-point scale ( $1=$ much less than others, $5=$ much more than others)
Group work confidence and prior experience measures
Demographic information

2. Mid-Course Survey (Week 4) - Progress Assessment

Current marketing familiarity: Same 5-point scale for paired comparison
Actual contribution patterns: Same 5-point scale for expectation vs reality analysis
Collaboration quality ratings (5-point scales)
Group member satisfaction (5-point scales)
Confidence growth compared to Week 1
Learning impact: How group work affected understanding
Most helpful learning methods (multiple choice)
Benefits and challenges of group work (multiple choice, multiple responses allowed)
Group vs individual case study preferences

3. Performance Metrics

Group case study scores ( 4 assignments $\times 100$ points each)
Individual exam scores ( 3 exams $\times 200$ points each, best 2 counted)
Final course performance metrics

Sample Characteristics

Enrollment and Response:

Initial enrollment: 70 students
Stayed through course: 67 students
Pre-survey respondents: 62 students * Mid-survey respondents: 55 students
Matched dataset: 50 students completed BOTH surveys (used for all analyses)

PRIMARY FINDINGS: Significant Learning Outcomes

Finding 1: Highly Significant Marketing Knowledge Growth

Marketing Concept Familiarity: PRE vs MID Comparison ( $\mathbf{n} \boldsymbol{=} \mathbf{5 0}$ matched students)

01_marketing_familiarity_comparison.png

Pre-Survey (Week 1): * Mean = $2.48(\mathrm{SD}=0.88)$ * Range: 1-4 on 5-point scale * Distribution: - Not familiar at all (1): 6 students (12\%) - Slightly familiar (2): 19 students (38\%) - Moderately familiar (3): 20 students (40\%) - Very familiar (4): 5 students (10\%) - Extremely familiar (5): 0 students (0\%)

Mid-Survey (Week 4): * Mean $=3.48(\mathrm{SD}=0.80)$ * Range: 2-5 on 5-point scale * Distribution: - Slightly familiar (2): 3 students (6\%) - Moderately familiar (3): 23 students (46\%) - Very familiar (4): 21 students (42\%) - Extremely familiar (5): 3 students (6\%)

Statistical Analysis: * Change: $+\mathbf{4 0 . 3 \%}$ improvement * Paired t-test: $\mathbf{t}(\mathbf{4 9})=\mathbf{7 . 8 2 7}, \mathbf{p}<$ 0.001 * Effect Size: Cohen’s d = 1.11

Conclusion: HIGHLY SIGNIFICANT

Students showed substantial, statistically significant improvement in marketing concept understanding after 4 weeks of collaborative case study learning. This represents the strongest evidence of teaching effectiveness in the study.

Finding 2: Contribution Pattern Recalibration

Expected vs Actual Contribution Patterns ( $\mathbf{n} \boldsymbol{=} \mathbf{5 0}$ matched students)

02_contribution_patterns_comparison.png

Pre-Survey (Expected Contributions): * Mean $=3.58(\mathrm{SD}=0.61) *$ Distribution: - About equally to others (3): 24 students (48\%) - More than others (4): 23 students (46\%) - Much more than others (5): 3 students (6\%)

Mid-Survey (Actual Contributions): * Mean = 3.36 ( $\mathrm{SD}=0.66$ ) * Distribution: - Less than others (2): 1 student (2\%) - About equally to others (3): 33 students (66\%) - More than others (4): 13 students ( $26 \%$ ) - Much more than others (5): 3 students (6\%)

Change: -0.22 points

Interpretation:

Students initially expected to contribute “more than others” but adjusted to realistic assessments of contributing “about equally” after actual collaborative experience.

MID-COURSE COLLABORATIVE LEARNING OUTCOMES (Week 4, $\mathrm{n}=50$ )

Outcome 1: High-Quality Collaboration

03_group_collaboration_quality.png

Collaboration Quality Rating: * Mean = 4.18/5.0 ( $\mathrm{SD}=0.99$ ) * Distribution: - Excellent (5): 25 students (50\%) - Good (4): 13 students (26\%) - Fair (3): 9 students (18\%) - Poor (2): 2 students (4\%) - Very Poor (1): 1 student (2\%)

Group Member Satisfaction: * Mean = 4.26/5.0 (SD = 0.99) * Distribution: - Very satisfied (5): 29 students (58\%) - Satisfied (4): 9 students (18\%) - Neutral (3): 9 students (18\%) Dissatisfied (2): 2 students (4\%) - Very dissatisfied (1): 1 student (2\%)

Interpretation: 76\% of students rated collaboration quality as “Good” or “Excellent,” and 76\% were satisfied or very satisfied with group members, indicating effective group functioning for the majority.

Outcome 2: Positive Impact on Learning

04_group_work_impact_on_learning.png

How Group Work Affected Understanding: * Significantly helped my learning: 7 students (14\%) * Somewhat helped my learning: 34 students (68\%) * Total reporting positive impact: 41 students (82\%) * Had no effect on my learning: 8 students (16\%) * Somewhat hindered my learning: 1 student (2\%) * Significantly hindered my learning: 0 students (0\%)

Interpretation: More than 4 out of 5 students perceived that collaborative learning positively impacted their understanding of marketing concepts, providing strong evidence of pedagogical effectiveness.

Outcome 3: Widespread Confidence Growth

Confidence in Analyzing Marketing Cases (Week 4 vs Week 1): * Much more confident now: 10 students (20\%) * More confident now: 37 students (74\%) * Total confidence increase: 47 students (94\%) * About the same: 3 students (6\%) * Less confident now: 0 students (0\%) * Much less confident now: 0 students (0\%)

Interpretation: Nearly all students (94\%) reported increased analytical confidence, with no students reporting decreased confidence. This demonstrates successful development of selfefficacy in marketing case analysis.

Outcome 4: Effective Learning Methods

06_most_helpful_learning_methods.png

Most Helpful Learning Methods (Multiple responses allowed): 1. Applying marketing concepts to real companies: 15 students ( $30 \%$ ) 2 . Group case study discussions: 11 students ( $22 \%$ ) 3. Individual reflections on case studies: 8 students ( $16 \%$ ) 4. Reading the textbook: 8 students ( $16 \%$ ) 5 . Working with group members on assignments: 7 students ( $14 \%$ )

Interpretation: Application-based learning and collaborative discussions emerged as most valuable, validating the case study approach.

Outcome 5: Student Learning Format Preferences

07_group_vs_individual_preference.png

Preference: Group vs Individual Case Studies: * Strongly prefer working on group case studies: 11 students (22\%) * Somewhat prefer working on group case studies: 11 students (22\%) * Total preferring group work: 22 students ( $\mathbf{4 4 \%}$ ) * No preference: 12 students ( $24 \%$ ) * Somewhat prefer working on individual case studies: 8 students ( $16 \%$ ) * Strongly prefer working on individual case studies: 8 students ( $16 \%$ )

Among students with a preference ( $\mathbf{n}=\mathbf{3 8}, \mathbf{1 1}+\mathbf{1 1}+\mathbf{8 + 8}$ ): $58 \%$ prefer group case studies Interpretation: Despite the challenges of collaborative work, students with preferences favored group over individual case studies by a 58:42 margin.

Outcome 6: Multiple Benefits of Collaboration

08_benefits_of_group_work.png

Benefits of Group Work (Multiple responses allowed): 1. Sharing the workload: 38 students ( $76 \%$ ) 2. Getting different perspectives on the same problem: 31 students (62\%) 3. Learning from others’ knowledge and skills: 28 students ( $56 \%$ ) 4 . Catching mistakes I might miss alone: 23 students ( $46 \%$ ) 5. Improving my communication skills: 21 students ( $42 \%$ ) 6. Learning to work with diverse people: 19 students ( $38 \%$ ) 7. Building professional networking skills: 12 students ( $24 \%$ ) 8 . Making friends: 8 students ( $16 \%$ )

Key Insights: * Learning benefits (perspectives, knowledge sharing, error detection) were valued alongside practical benefits (workload sharing) * 62\% valued diverse perspectives, supporting the pedagogical goal of perspective-taking * 56\% reported learning from others’ knowledge, confirming peer learning occurred * 42\% saw communication skill development, an important professional competency

Outcome 7: Identified Challenges

09_challenges_of_group_work.png

Challenges of Group Work (Multiple responses allowed): 1 . Coordinating schedules with group members: 20 students ( $40 \%$ ) 2. Communication difficulties: 20 students ( $40 \%$ ) 3. Unequal contribution from group members: 17 students ( $34 \%$ ) 4. Managing time effectively: 17 students ( $34 \%$ ) 5. Different standards for work quality: 13 students ( $26 \%$ ) 6. Conflicts within the group: 7 students (14\%) 7. Technology issues (e.g., sharing documents): 6 students (12\%) 8. Lack of trust among members: 3 students (6\%)

Key Insights: * Coordination challenges (scheduling, communication) were most common ( $40 \%$ each) * Contribution equity concerns affected one-third of students ( $34 \%$ ) * Time management challenges ( $34 \%$ ) important for intensive summer course * Trust issues were rare (6\%), suggesting generally positive group dynamics

Design Implications: These challenges identify specific areas for instructional support: such as scheduling tools, communication guidelines, contribution monitoring systems, and time management resources. I will try my best to done this better next summer.

Collaborative Problem-Solving Skills Development: Evidence Dashboard [Matched Students $\mathbf{n = 5 0}$ ]

Performance Outcomes and Assessment Design Discovery Group Work Performance ( $\mathrm{n}=50$ ) Group Case Study Scores: * Mean = 98.55\% (SD = 1.15\%) * Range: 96.25\% - 100\% * Coefficient of Variation $=1.17 \%$ (extremely low variability)

Assessment Finding: Clear ceiling effect evident - assessment was too lenient to discriminate among different levels of group performance.

Individual Exam Performance ( $\mathrm{n}=50$ ) Individual Exam Scores: * Mean $=91.86 \%(\mathrm{SD}=3.05 \%) *$ Range: $81 \%-98 \% *$ Coefficient of Variation $=3.32 \%($ normal academic variability $) * 7 \times$ higher variance compared to group work

Performance Distribution: Group Work vs Individual Exams

Performance Gap Analysis

Group vs Individual Differential: * Group work scores systematically 6.7 percentage points higher than individual exam scores * Performance ceiling in collaborative assessment prevented meaningful variance * Individual assessments showed normal distribution and discrimination

Critical Methodological Insight

The ceiling effect in group work assessment prevented using performance scores to validate learning outcomes. However, the comprehensive mid-course survey captured learning that performance scores alone could not measure:

$40.3 \%$ improvement in marketing knowledge ( $\mathrm{p}<0.001$ )
$94 \%$ gained analytical confidence
$82 \%$ reported learning benefits
High collaboration quality ( $\mathrm{M}=4.18 / 5.0$ ) and satisfaction ( $\mathrm{M}=4.26 / 5.0$ )

Conclusion: Multi-dimensional assessment (performance + attitudes + self-reported learning + confidence) provides more valid evidence of teaching effectiveness than performance scores alone.

Key Insights and Pedagogical Implications What the Data Demonstrates

1. Collaborative Case Study Learning Works

The evidence strongly supports collaborative learning effectiveness: * Large, significant knowledge gains: $40.3 \%$ improvement ( $\mathrm{p}<0.001$, Cohen’s $\mathrm{d}=1.11$ ) * Confidence development: $94 \%$ of students more confident in analysis * Perceived learning value: $82 \%$ reported group work helped their learning * High collaboration quality: Mean 4.18/5.0 across groups

2. Students Experience Multiple Benefits

Learning occurred across multiple dimensions: * Content mastery: Marketing concept familiarity increased significantly * Cognitive skills: Analytical confidence and case analysis abilities improved * Collaborative capabilities: Diverse perspective-taking, knowledge sharing, error detection * Professional skills: Communication, teamwork, working with diverse people

3. Challenges Are Real But Manageable

Students identified specific obstacles while still valuing the experience: * $40 \%$ faced scheduling/communication challenges * $34 \%$ noted contribution equity concerns * Despite challenges, $58 \%$ (of those with preferences) still preferred group work * High satisfaction ( $M=4.26 / 5.0$ ) suggests challenges didn’t undermine collaboration

4. Assessment Design Critically Important

The ceiling effect discovery has major implications: * Group performance scores ( $\mathrm{M}=98.55 \%$, $\mathrm{SD}=1.15 \%$ ) couldn’t capture learning variance * Comprehensive surveys revealed learning outcomes performance scores missed * Multi-method assessment essential for validating collaborative learning * Need for rigorous rubrics that maintain discrimination while supporting collaboration

Implications for Future Teaching Practice

Immediate Changes:

Revise Group Assessment Rubrics * Increase rigor to create meaningful performance variance * Develop criteria capturing individual learning within group context * Align difficulty with individual exam standards * Target mean around $85-90 \%$ instead of $98.55 \%$
Enhance Individual Accountability * Implement peer evaluation systems * Include individual components within group projects * Use collaboration tracking tools (Google Docs edit history) * Consider individual oral defenses or reflections
Address Identified Challenges * Provide structured scheduling tools and communication guidelines * Offer explicit training on managing unequal contributions * Build in checkpoint meetings for time management * Create clearer expectations for work quality standards
Expand Assessment Methods * Continue comprehensive surveys capturing learning beyond performance * Add qualitative reflections on collaboration experiences * Implement pre-midpost design to track learning trajectories * Assess multiple outcomes: knowledge, skills, attitudes, processes

What I Learned

Primary Discovery: Collaborative learning in marketing case studies produces significant, measurable improvements in student learning outcomes. The matched-student analysis ( $\mathrm{n}=50$ ) revealed:

Strong Knowledge Development: Students showed a $40.3 \%$ improvement in marketing concept familiarity ( $\mathrm{p}<0.001, \mathrm{~d}=1.11$ ), demonstrating that collaborative case study analysis effectively builds content mastery.
Confidence Building: $94 \%$ of students reported increased confidence in analyzing marketing cases, suggesting the pedagogical approach successfully develops analytical capabilities.
High-Quality Collaboration: Mean collaboration quality of $4.18 / 5.0$ and member satisfaction of 4.26/5.0 indicate that most groups functioned effectively, creating positive learning environments.
Perceived Learning Value: $82 \%$ of students reported that group work helped their learning, with specific benefits including diverse perspectives ( $62 \%$ ), knowledge sharing ( $56 \%$ ), and error detection ( $46 \%$ ).
Assessment Design Matters: The ceiling effect in group performance scores ( $\mathrm{M}=98.55 \%, \mathrm{SD}=1.15 \%$ ) masked individual differences, but the comprehensive midcourse survey captured learning outcomes that performance scores alone could not measure.

What Surprised Me

Magnitude of Knowledge Growth: The $40.3 \%$ improvement in marketing familiarity ( $\mathrm{p}<0.001$ ) exceeded expectations, showing that collaborative learning can produce large, statistically significant learning gains in a short timeframe.
Realistic Self-Assessment Development: Students adjusted their contribution expectations ( $M=3.58$ ) to match reality ( $M=3.36$ ), showing healthy recalibration rather than inflated self-perception.
Positive Attitudes Despite Challenges: Even while identifying real challenges ( $40 \%$ cited scheduling/communication issues), $82 \%$ still reported learning benefits and $58 \%$ (of those with preferences) preferred group over individual case studies.
Multiple Learning Dimensions: The data revealed learning occurred across multiple dimensions - not just content knowledge, but also confidence, collaboration skills, and perspective-taking abilities.
Assessment Limitations Revealed Through Multiple Measures: Performance scores alone (with ceiling effects) would have suggested little learning occurred, but the comprehensive survey data revealed substantial development across multiple dimensions.

What I Would Do Differently

Immediate Changes for Future Course Offerings:

1. Revise Group Assessment Rubric:

Increase rigor and discrimination to create meaningful performance variance
Develop scoring criteria that capture individual learning within group context
Align assessment difficulty with individual exam standards
Target mean around $85-90 \%$ instead of $98.55 \%$

2. Enhance Individual Accountability:

Implement peer evaluation systems to capture individual contributions
Include individual components within group projects
Use Google Docs tracking to monitor actual contribution patterns
Consider individual oral defenses or written reflections on group projects

3. Expand Assessment Methods:

Continue using comprehensive surveys to capture learning beyond performance scores
Add qualitative reflections on collaboration experiences
Implement pre-mid-post design to track learning trajectory
Assess multiple learning outcomes: knowledge, skills, and attitudes

Implications for Future Classroom Practice

Evidence-Based Teaching Strategy: This research provides concrete evidence that collaborative marketing case study learning produces measurable improvements in:

Marketing concept understanding ( $+40.3 \%, \mathrm{p}<0.001$ )
Analytical confidence ( $94 \%$ increase)
Collaborative capabilities (high quality ratings)
Multiple skill dimensions (communication, perspective-taking, error detection)

Key Pedagogical Shift:

FROM: Questioning whether collaborative learning “works”
TO: Understanding how to assess collaborative learning validly

The data clearly demonstrates that collaborative case study learning is pedagogically effective, but assessment design must capture learning across multiple dimensions rather than relying solely on group performance scores.

Broader Impact:

This project demonstrates that systematic teaching inquiry can reveal:

What works: Collaborative case study learning produces significant learning gains
What needs improvement: Assessment design must match learning complexity
How to measure success: Multiple outcome measures capture learning performance scores miss

The finding of $40.3 \%$ improvement in marketing knowledge ( $\mathrm{p}<0.001, \mathrm{~d}=1.11$ ) provides compelling evidence for the value of collaborative learning in marketing education, while the identified assessment limitations guide future improvement efforts.

Future Research Questions:

How can group work rubrics maintain rigor while supporting collaborative learning?
What individual accountability measures optimize both learning and assessment validity?
How do collaborative learning outcomes develop over longer timeframes (full semester)?
What specific instructional practices within groups maximize learning gains?

Conclusion

This mentored teaching project provides strong empirical evidence that collaborative marketing case study learning produces significant, measurable improvements in student learning outcomes. The matched-student analysis ( $\mathrm{n}=50$ ) revealed:

40.3\% improvement in marketing concept familiarity ( $\mathrm{p}<0.001$, Cohen’s $\mathrm{d}=1.11$ )
$\mathbf{9 4 \%}$ of students gained analytical confidence
82\% reported learning benefits from collaboration
High collaboration quality ( $M=4.18 / 5.0$ ) and satisfaction ( $M=4.26 / 5.0$ )

These findings validate the pedagogical effectiveness of collaborative learning while identifying specific areas for improvement (assessment rubric rigor, individual accountability systems). The research demonstrates how systematic teaching inquiry can both validate effective practices and reveal methodological limitations requiring attention.

Key Takeaway: Collaborative case study learning clearly works for developing marketing knowledge and analytical capabilities, but assessing it validly requires multi-dimensional measurement beyond performance scores alone. Future iterations will maintain the proven collaborative learning approach while implementing more rigorous, discriminating assessment methods that capture both individual learning and collaborative skill development.

This evidence-based approach to teaching improvement - asking focused questions, gathering multiple forms of evidence, analyzing rigorously, and applying findings to future practice represents the core of the scholarship of teaching and learning, fulfilling the requirements of Competency 5.

Minghao Wang