educational-research-methods

# Educational Research Methods

A comprehensive skill for conducting rigorous educational research using both quantitative and qualitative methodologies. Covers study design, data collection instruments, analysis techniques, and reporting standards specific to education scholarship.

## Study Design Frameworks

### Quantitative Designs

Educational quantitative research typically follows one of these designs:

| Design | Purpose | Example |
|--------|---------|---------|
| Randomized controlled trial (RCT) | Causal inference | Random assignment to instruction methods |
| Quasi-experimental | Causal inference without randomization | Pre-post comparison with matched control |
| Correlational | Relationship exploration | Survey linking self-efficacy to GPA |
| Longitudinal panel | Change over time | Tracking cohort achievement K-12 |
| Cross-sectional survey | Snapshot description | National teacher satisfaction survey |

### Qualitative Designs

Common qualitative traditions in education:

- **Ethnography**: Extended immersion in a classroom or school culture to produce thick description
- **Case study**: In-depth examination of a bounded system (a program, a school, a student)
- **Grounded theory**: Iterative coding to build theory from interview and observation data
- **Phenomenology**: Exploring the lived experience of participants (e.g., first-generation college students)
- **Action research**: Practitioners systematically studying their own practice to improve it

### Mixed Methods

Sequential and concurrent mixed-methods designs are increasingly common in education research:

```
Sequential Explanatory:
  Phase 1: Quantitative survey (n=500) --> identify patterns
  Phase 2: Qualitative interviews (n=20) --> explain patterns

Concurrent Triangulation:
  QUAN data collection + QUAL data collection (simultaneous)
  --> merge and compare findings at interpretation stage

Embedded Design:
  Primary: RCT measuring learning outcomes
  Secondary: Classroom observations embedded within treatment arm
```

## Data Collection Instruments

### Survey and Questionnaire Design

```python
import pandas as pd
from scipy import stats

# Reliability analysis for a Likert-scale instrument
def cronbach_alpha(df: pd.DataFrame) -> float:
    """
    Compute Cronbach's alpha for internal consistency reliability.
    df: DataFrame where each column is an item, each row a respondent.
    Acceptable threshold: alpha >= 0.70 for research purposes.
    """
    n_items = df.shape[1]
    item_vars = df.var(axis=0, ddof=1)
    total_var = df.sum(axis=1).var(ddof=1)
    alpha = (n_items / (n_items - 1)) * (1 - item_vars.sum() / total_var)
    return round(alpha, 4)

# Example usage with a 6-item motivation scale
data = pd.DataFrame({
    'item1': [4, 5, 3, 4, 5, 3, 4, 5],
    'item2': [3, 4, 3, 4, 5, 2, 4, 4],
    'item3': [4, 5, 4, 5, 4, 3, 5, 5],
    'item4': [3, 4, 2, 3, 5, 2, 3, 4],
    'item5': [4, 5, 3, 4, 5, 3, 4, 5],
    'item6': [3, 4, 3, 4, 4, 3, 4, 4],
})

alpha = cronbach_alpha(data)
print(f"Cronbach's alpha: {alpha}")
# alpha >= 0.70 indicates acceptable internal consistency
```

### Observation Protocols

Structured classroom observation instruments:

- **CLASS (Classroom Assessment Scoring System)**: Measures teacher-student interactions across emotional support, classroom organization, and instructional support
- **RTOP (Reformed Teaching Observation Protocol)**: Evaluates inquiry-based instruction in STEM
- **Flanders Interaction Analysis**: Codes teacher talk, student talk, and silence in timed intervals

### Interview Protocols

Semi-structured interview best practices for educational research:

1. Begin with rapport-building questions before moving to core topics
2. Use open-ended prompts: "Tell me about..." rather than yes/no questions
3. Prepare follow-up probes for each core question
4. Pilot the protocol with 2-3 participants and revise
5. Plan for 45-60 minute sessions to allow depth without fatigue

## Analysis Techniques

### Quantitative Analysis for Education Data

```python
import statsmodels.api as sm
from statsmodels.formula.api import mixedlm

# Hierarchical Linear Model (HLM) -- essential for nested
# education data (students within classrooms within schools)
# Example: predicting math achievement from student SES
# and classroom teaching quality

model = mixedlm(
    "math_score ~ student_ses + teaching_quality",
    data=df,
    groups=df["school_id"],
    re_formula="~teaching_quality"
)
result = model.fit()
print(result.summary())

# Effect size calculation (Cohen's d)
def cohens_d(group1, group2):
    n1, n2 = len(group1), len(group2)
    var1, var2 = group1.var(), group2.var()
    pooled_std = ((( n1 - 1) * var1 + (n2 - 1) * var2) / (n1 + n2 - 2)) ** 0.5
    return (group1.mean() - group2.mean()) / pooled_std
```

### Qualitative Coding

Thematic analysis workflow (Braun and Clarke, 2006):

1. **Familiarization**: Read transcripts multiple times, take initial notes
2. **Initial coding**: Generate codes systematically across the dataset
3. **Theme search**: Collate codes into candidate themes
4. **Theme review**: Check themes against coded extracts and full dataset
5. **Theme definition**: Refine names and write analytic narrative
6. **Report**: Select vivid, compelling quotes that capture each theme

Tools: NVivo, ATLAS.ti, MAXQDA, or open-source Taguette for coding.

## Reporting Standards

### APA and AERA Guidelines

Educational research follows the APA Publication Manual (7th edition) and the AERA Standards for Reporting on Empirical Social Science Research:

- Report effect sizes alongside p-values for all statistical tests
- Describe the sample demographics in detail (age, gender, race/ethnicity, SES)
- Discuss both statistical significance and practical significance
- For qualitative work, describe researcher positionality and reflexivity
- Include limitations section addressing threats to validity

### Key Journals

- *American Educational Research Journal* (AERJ)
- *Educational Researcher*
- *Journal of Educational Psychology*
- *Review of Educational Research*
- *Teaching and Teacher Education*
- *International Journal of Educational Research*

## Ethical Considerations

Educational research involving human subjects (especially minors) requires Institutional Review Board (IRB) approval. Key considerations include informed consent from parents/guardians, assent from minors, data de-identification, and equitable participant selection. The Belmont Report principles (respect for persons, beneficence, justice) guide all education research ethics.