GPT-4, OpenAI’s newest version of ChatGPT, “passes a simulated bar exam with a score around the top 10% of test takers; in contrast, GPT-3.5’s score was around the bottom 10%,” according to OpenAI. This represents a tremendous leap in the LLM’s capability in this area. OpenAI also made huge gains in AP exams and the LSAT, among others, with its latest GPT version.

And GPT-4 also pushed the LLM over or very near the 90th percentile on other common academic tests:

• SAT math: 89th percentile

• GRE verbal: 99th percentile

• AP statistics, macroeconomics, biology: all around 90th percentile

OpenAI tested its LLM “by using the most recent publicly-available tests (in the case of the Olympiads and AP free response questions) or by purchasing 2022–2023 editions of practice exams.” They “did no specific training for these exams.”

The company also claims GPT-4 “considerably outperforms” benchmarks designed to assess the effectiveness of LLMs, scoring higher in reasoning, Python coding, reading comprehension, and arithmetic, among a few others.

-Brandon Gorrell