# ChatGPT for statistical analysis: when to use it, what to check, and the alternatives

> Use ChatGPT to explore data and draft analysis code, then check the test, the defaults and the numbers before you publish. Here is what published evaluations found, a checklist, and when another tool fits better.

Canonical: https://www.licklider.ai/alternatives/chatgpt-for-statistical-analysis/
Section: Alternatives › AI data-analysis assistants
Author: Licklider · Published 2026-11-10 · Last reviewed 2026-11-10 (UTC)

> **Disclosure.** Licklider makes SciGround, one of the options on this page. Options are grouped by the job they do rather than ranked, and each statement about a product names its source and the date we checked it.

> **Quick answer.** Use ChatGPT to explore a dataset, draft analysis code and explain output. For a result you will publish, treat its analysis as a draft: give it your design, ask it to name the test and every default it used, and rerun the code yourself, or check it with a verification tool for the analyses that tool covers. In a 2024 evaluation of GPT-4, results of inferential tests matched the expected values in 32.5% of attempts with basic prompts and in 92.5% when the prompt also gave cleaning and assumption steps and named the test ([Ruta et al. 2025](https://doi.org/10.2196/63550)); newer models were not tested.

> **Terms used below.** *Inferential test*: a test that produces a p-value or confidence interval, such as a t-test. *Student's t-test*: assumes both groups have the same spread. *Welch's t-test*: does not assume that, and is the default this site recommends. *Degrees of freedom (df)*: a number printed with the test that reflects the amount of independent data. *Yates' correction*: an adjustment to the chi-square test that makes its p-value larger (more conservative). *Default argument*: a setting a function uses unless you change it. *MCP*: the standard that lets an AI assistant connect to an outside tool.

## What ChatGPT does well for statistics

In the 2024 evaluation by Ruta and colleagues, GPT-4 ran Python code on an uploaded file, so it could clean data, make tables and run tests in one conversation. Asked to process a subset of the US National Inpatient Sample, it **reclassified and subset variables, built tables and computed means, SDs, medians and IQRs correctly in every trial** ([Ruta et al. 2025](https://doi.org/10.2196/63550), Results).

Those results support three jobs:

- **Exploring** a new dataset: summaries, plots, "what does this column contain?".
- **Drafting analysis code** that you then read, run and keep.
- **Explaining** output you already have, such as what a confidence interval or a Welch correction means.

The authors' own conclusion is the right frame: promising for exploratory analysis, with accuracy that depends on careful prompts and human oversight. Use it for those jobs, and keep the code it writes.

## Where it went wrong in published evaluations, and what to do about each

In these 2024 evaluations, errors clustered in **choosing and running the inferential test** rather than in arithmetic. Each finding below comes with the step that prevents it.

| Finding | Evidence (model, date) | What to do |
| --- | --- | --- |
| With a basic prompt (question and variables only), the method was right in 47.5% of attempts (38/80) and the result matched in 32.5% (26/80). For the two-sample t-test, the chosen method did not match the authors' expected test (Student's t-test) in 10 of 10 attempts, and for Kruskal–Wallis in 10 of 10; the paper does not report what was chosen instead, and under this scoring a Welch's t-test (this site's default for unequal spreads) would also count as a mismatch. | Ruta et al. 2025, Tables 4–5; GPT-4, data collected before June 2024 | Describe the design (groups, what one independent unit is, paired or not) and ask it to justify the test against that design. |
| Adding data-cleaning steps and assumption checks raised method choice to 85%; also naming the test raised every measure to 92.5% (74/80). | Ruta et al. 2025, Table 4 | Decide the test yourself, using a guide or the [test chooser](/tools/statistical-test-chooser/), and put it in the prompt. |
| All ten basic-prompt chi-square attempts gave a different value because SciPy's function applies Yates' continuity correction by default, which the model kept for a large sample (R's default does the same; the study scored it as a coding error). | Ruta et al. 2025, Table 5 and Results | Ask it to list every default argument it relied on, and set them explicitly. |
| On the StatQA benchmark of 11,623 statistical tasks, GPT-4o's best score was 64.83%; the models mostly made *applicability* errors, choosing a method whose conditions do not hold. | [Zhu et al. 2024](https://arxiv.org/abs/2406.07815), NeurIPS Datasets and Benchmarks | Ask for the conditions the method needs and check each one against how the data were collected. |

These studies tested GPT-4 and GPT-4o in 2024. Newer models may do better. Use the checklist below whichever model you have.

## Worked check: two defaults that change the number you report

You can see the default problem without ChatGPT at all, because it lives in the libraries the generated code calls. We ran the same constructed data through SciPy and R (data in the code below; a different dataset from the one in the Welch guide).

**Two groups with unequal spread and size** (control n = 10, treated n = 5, treated values far more variable):

| Call | Test actually run | Result |
| --- | --- | --- |
| `scipy.stats.ttest_ind(treated, control)` | Student's t-test (SciPy's default, `equal_var=True`) | t(13) = 2.04, p = 0.062 |
| `scipy.stats.ttest_ind(treated, control, equal_var=False)` | Welch's t-test | t(4.07) = 1.41, p = 0.23 |
| `t.test(treated, control)` in R | Welch's t-test (R's default) | t(4.07) = 1.41, p = 0.23 |

The same request, "compare the two groups with a t-test", gives p = 0.062 or p = 0.23 depending on the language the code was written in. With unequal spreads and sizes, Welch's result is the one to report ([Welch's t-test vs Student's t-test](/guides/welch-vs-student-t-test/)).

**A large 2 × 2 table** (120/80 vs 95/105): `chi2_contingency(table)` in SciPy and `chisq.test(tab)` in R both apply Yates' correction by default and give χ²(1) = 5.79, p = 0.016; with the correction switched off, χ²(1) = 6.29, p = 0.012. Here the conclusion survives, but the number in your paper depends on a setting the library chose for you.

```python
import numpy as np
from scipy import stats

control = np.array([4.1, 4.4, 3.9, 4.6, 4.2, 4.0, 4.3, 4.5, 4.1, 4.4])
treated = np.array([5.9, 3.1, 7.2, 4.0, 6.5])
print(stats.ttest_ind(treated, control))                         # SciPy default: Student's t-test
print(stats.ttest_ind(treated, control, equal_var=False))        # say which t-test you want: Welch

table = np.array([[120, 80], [95, 105]])
print(stats.chi2_contingency(table))                             # default: Yates' correction
print(stats.chi2_contingency(table, correction=False))           # say whether you want Yates
```

```r
control <- c(4.1, 4.4, 3.9, 4.6, 4.2, 4.0, 4.3, 4.5, 4.1, 4.4)
treated <- c(5.9, 3.1, 7.2, 4.0, 6.5)
t.test(treated, control)                     # R default: Welch's t-test
t.test(treated, control, var.equal = TRUE)   # Student's t-test

tab <- matrix(c(120, 95, 80, 105), nrow = 2)  # rows: group 1, group 2; columns: yes, no
chisq.test(tab)                              # default: Yates' correction
chisq.test(tab, correct = FALSE)
```

**What to do:** in any AI-written analysis, search the code for the test function and check its default arguments in the library documentation before you copy a number.

## Which tool for which job

| You need to… | A good fit | Why |
| --- | --- | --- |
| Explore a new dataset or draft code quickly | ChatGPT or another AI assistant | Conversational; in a 2024 evaluation GPT-4 ran the cleaning and descriptive steps correctly in every trial (Ruta et al.) |
| Produce a result for a paper with code you can rerun | R or Python, with the code saved | Free, scriptable and citable by version |
| Analyse and make publication graphs by pointing and clicking | GraphPad Prism (commercial), JASP or jamovi (free) | Menus instead of code; graphing built in |
| Check that an AI-produced analysis supports the claim you want to make | SciGround (our product), used inside ChatGPT or Claude | Verified results for its supported analyses only; contingency tables, Kruskal–Wallis, multi-factor designs and mixed models are not covered (limits below) |
| Work with no software budget | R, Python, JASP or jamovi | All free |

## The options, grouped by job

Each row states its source and the date we checked it. Nothing here comes from hands-on testing.

| Option | Best for | Less suited for | Basis (checked 10 Nov 2026) |
| --- | --- | --- | --- |
| **ChatGPT with data analysis** | Exploring data, drafting and explaining code | Final inferential results without your own check | Ruta et al. 2025 (GPT-4); Zhu et al. 2024 (GPT-4o) |
| **R** | Any analysis you want to rerun and cite; the reference for most methods on this site | Readers who will not write code | [r-project.org](https://www.r-project.org/): "a free software environment for statistical computing and graphics" |
| **Python (SciPy, statsmodels)** | Analyses inside a wider data pipeline | Readers who will not write code; check defaults (above) | Our reproducible test with SciPy 1.18.1 |
| **JASP** | Point-and-click analysis, free, in classical and Bayesian form, with APA-format tables | Analyses outside its menus | [jasp-stats.org](https://jasp-stats.org/): "a free cross-platform software program", open source, supported by the University of Amsterdam |
| **jamovi** | Point-and-click analysis that shows the R code; 70+ add-on modules; also runs in the browser | Analyses outside its modules | [jamovi.org](https://www.jamovi.org/): "a free and open statistical spreadsheet"; shows "the exact R syntax behind every analysis" |
| **GraphPad Prism** | Point-and-click analysis and publication graphs, with a broad library of tests; each analysis includes an assumptions checklist to confirm the test fits | No-budget settings (commercial licence; free trial offered) | [graphpad.com/features](https://www.graphpad.com/features): t-tests, "one-, two-, three-way and multifactor (N-way) ANOVA", curve fitting and graphing |
| **SciGround** (our product) | Checking analyses and claims inside ChatGPT or Claude, with a shareable record | Analyses outside its supported workflows; exploration in the chat, which stays unverified (SciGround's own exploration is recorded but not issued as confirmatory evidence) | Our capability pages: [analyse and verify](/sciground/analysis/), [supported analyses](/sciground/analyses/) and [their limits](/sciground/analyses/limits/) |

**What SciGround adds to ChatGPT, and where it stops.** SciGround connects to ChatGPT (web) and Claude (web and desktop) through MCP, with access details issued to you ([use with your AI](/sciground/connect/)); whether you can add a custom MCP server in ChatGPT depends on your account and workspace settings ([OpenAI developer documentation](https://developers.openai.com/apps-sdk/deploy/connect-chatgpt), checked 10 November 2026). Standard methods run on established libraries such as SciPy and NumPy with versions and settings recorded, an independent checker re-verifies the covered results, and each result comes with the claim it supports; SciGround rejects leaps such as reading non-significance as no effect ([analyse and verify](/sciground/analysis/)). Only results that pass through SciGround are verified: other answers in the same conversation are not ([limits](/sciground/connect/limits/)). The supported workflows are group comparisons (two groups, before and after, and one-way designs with three or more groups, analysed as a one-way ANOVA assuming equal variances), a two-group rank comparison (Mann–Whitney), several planned outcomes, bulk RNA-seq, measurements lost to equipment failure, three fixed batches, replications, two planned stopping decisions, and exploration followed by confirmation ([supported analyses](/sciground/analyses/)). Not covered: repeated measures beyond two time points, multi-factor designs and interactions, covariate adjustment, general mixed models, single-cell data and a general catalogue of non-parametric tests, which leaves out the chi-square and other contingency-table tests and Kruskal–Wallis used as examples above ([supported analyses: limits](/sciground/analyses/limits/)). For those, use R, Python or a menu-based package such as GraphPad Prism, whose features page lists Fisher's exact and chi-square tests, Kruskal–Wallis, two- and three-way ANOVA and mixed-effects models.

Read each product's own page before you decide; every row links to it.

## Use ChatGPT safely for statistics: a checklist

Copy the prompts and fill in the brackets. Before you upload anything, check your institution's rules for unpublished or identifiable data and the data settings of your ChatGPT account.

1. **Give the design, not just the columns.** *"Each row is one mouse; [n] mice per group; groups are independent; the outcome is [measure]. Which test fits, and why?"* If several rows come from one animal, say so, and read [what counts as n](/guides/what-counts-as-n/) first.
2. **Ask for the test and its settings by name.** *"Name the exact function, its sidedness and every default argument you rely on."* Then set them explicitly.
3. **Check the conditions against your design.** The common ones, in plain words:

   | Test | Needs |
   | --- | --- |
   | Welch's t-test (two independent groups) | One value per independent unit; roughly symmetric values without extreme outliers; unequal spreads are fine |
   | Paired t-test | Pairs from the same unit; roughly symmetric differences |
   | Mann–Whitney | Independent units; skewed or ordinal values are fine |
   | Welch's ANOVA (three or more groups) | As Welch's t-test, for each group |
   | Chi-square test | Counts of independent subjects; expected count of at least 5 in every cell, otherwise Fisher's exact test |

   Judge them from the design and a plot; a normality test on five values is too weak to settle them. The [test chooser](/tools/statistical-test-chooser/) covers more designs.
4. **Ask for an effect size and a confidence interval** as well as the p-value.
   **Count your tests.** If you compared more than two groups or analysed several outcomes, name the correction you will use before you look ([which correction to use](/guides/which-multiple-comparison-correction/)).
5. **Check what happened to the rows.** *"How many rows went in, how many were analysed, and were any removed, imputed or recoded?"* Ruta and colleagues recorded an outlier removal and an imputed value among the errors.
6. **Rerun the code yourself** in R or Python and compare every number with the chat's answer. If they differ, find the setting that differs (test, default, rows used), report the number from the analysis you can justify and rerun, and say which setting you chose; never keep the smaller p because it is smaller. Save the code with your data.
7. **Report the software and versions** that produced the final numbers, and follow your journal's policy on describing assistance from AI tools.

## When to choose another tool instead

- **Choose ChatGPT alone** for exploration, code drafts and explanations, where you will rerun anything that matters.
- **Choose R or Python directly, or a menu-based package such as Prism, JASP or jamovi,** when the analysis is outside SciGround's supported workflows (for example, contingency tables, two-way designs with interactions or mixed models); choose R or Python when you want full control of every setting.
- **Choose GraphPad Prism, JASP or jamovi** when you prefer menus to code or need publication graphs; Prism is commercial, JASP and jamovi are free.
- **Choose SciGround** when you already work in ChatGPT or Claude, your analysis is one of its supported workflows, and you need the result and the claim checked and shareable.

Whichever you pick, run the checklist above on the analysis before you report it.

## How we compared

- **Sources:** each product's own public page, checked on 10 November 2026; two published evaluations (Ruta et al. 2025, GPT-4, data collected in 2024; Zhu et al. 2024, GPT-4o and other models).
- **Our reproducible test:** library defaults in SciPy 1.18.1 and R 4.3.3 on constructed data, with stored code. It tests the libraries that generated code calls rather than any product.
- **OpenAI's product and help pages could not be read** by our checks on 10 November 2026, so current ChatGPT features, plans and data settings are not described here.
- **No hands-on testing** of ChatGPT, Prism, JASP, jamovi or any other product was done. The page's main value is the checklist, which works with any tool, including ours.
- **Conflict of interest:** Licklider makes SciGround.

Check any statement against its linked source and date.

## Questions

**Can ChatGPT do statistics correctly?** In a 2024 evaluation of GPT-4, it computed descriptive statistics correctly in every trial, and its inferential results matched the expected values in 32.5% to 92.5% of attempts depending on how specific the prompt was; newer models were not tested. Give it the design and the test, and rerun the code.

**Is it acceptable to use ChatGPT for the analysis in a paper?** Journal policies on AI assistance differ, so check yours before you submit. Whatever the policy, keep the code and report the software that produced the final numbers. Before you upload data to ChatGPT, check your institution's rules for unpublished or identifiable data and the data settings of your account.

**Does SciGround replace ChatGPT?** No. It works inside ChatGPT and Claude and verifies the results of its supported analyses that pass through it; exploration, other answers in the conversation and analyses outside its workflows (for example chi-square and other contingency-table tests, Kruskal–Wallis, multi-factor designs and mixed models) stay unverified.

## References

- Ruta MR, Gaidici T, Irwin C, Lifshitz J (2025). ChatGPT for univariate statistics: validation of AI-assisted data analysis in healthcare research. *Journal of Medical Internet Research* 27:e63550. [doi:10.2196/63550](https://doi.org/10.2196/63550)
- Zhu Y, Du S, Li B, Luo Y, Tang N (2024). Are large language models good statisticians? NeurIPS 2024 Datasets and Benchmarks. [arXiv:2406.07815](https://arxiv.org/abs/2406.07815)
- R Core Team. The R Project for Statistical Computing. [r-project.org](https://www.r-project.org/) (checked 10 November 2026)
- JASP Team. JASP. [jasp-stats.org](https://jasp-stats.org/) (checked 10 November 2026)
- The jamovi project. jamovi. [jamovi.org](https://www.jamovi.org/) (checked 10 November 2026)
- GraphPad Prism features. [graphpad.com/features](https://www.graphpad.com/features) (checked 10 November 2026)

## Changes

- 10 November 2026: first published.

*ChatGPT, Claude, GraphPad Prism, JASP, jamovi and Python are names of their owners' products. Licklider is not affiliated with OpenAI, Anthropic, GraphPad, the JASP team, the jamovi project or the Python Software Foundation.*

## Related

- [Statistical test chooser](https://www.licklider.ai/tools/statistical-test-chooser/)
- [Welch's t-test vs Student's t-test](https://www.licklider.ai/guides/welch-vs-student-t-test/)
- [SciGround: use with your AI](https://www.licklider.ai/sciground/connect/)
- [Claude, Gemini or Copilot for statistics](https://www.licklider.ai/alternatives/ai-assistants-for-statistical-analysis/)
- [Julius AI alternatives](https://www.licklider.ai/alternatives/julius-ai-alternatives/)
- [How to check an AI assistant's statistical analysis](https://www.licklider.ai/guides/how-to-check-an-ai-statistical-analysis/)
- [How to disclose an AI assistant's role in a statistical analysis](https://www.licklider.ai/guides/how-to-disclose-ai-use-in-statistical-analysis/)

## How this page was made

- Written and published by Licklider, which makes SciGround, one of the options compared here.
- Statements about other products come from their own public pages, checked on 10 November 2026, and from two published evaluations whose model and date are given in the text. No hands-on testing of any product was done.
- The worked check is a reproducible test of library defaults, run with stored code (Python 3.13 with SciPy 1.18.1; R 4.3.3); both languages give the same numbers.
- Separate review passes checked the statistics, the citations, the product and comparative statements, and readability before publication.