AI can produce a convincing engineering answer. But can it read an entire project folder, spot an outdated drawing, follow building codes, and deliver a design that a licensed engineer would approve?
Google and Mercor have released a new benchmark designed to find out. Announced on November 3, 2026, Physical-Engineering’s Last Exam (PLE) challenges AI agents with 85 realistic assignments spanning mechanical, electrical, civil, and structural engineering.
The initial results are striking. Across the models evaluated, none reached a 50% mean score on PLE’s 56-task hidden test set. The leading model-and-agent configuration scored 48.3%.
That does not mean AI fails at half of all engineering work. It means the systems tested struggled on this particular set of complex, end-to-end assignments, where a plausible answer is not enough.
A test built around real engineering work
Traditional AI tests often ask for a short answer or a single calculation. PLE is closer to handing an engineer a project package and asking for a deliverable that can survive professional review.
Each task provides a simulated project environment containing documents an engineer would routinely encounter, including technical drawings, emails, spreadsheets, PDFs, and revision histories. Across the benchmark, there are 5,170 files, averaging about 61 files per assignment.
An agent might need to identify which drawing is current, interpret design constraints, calculate loads or capacity, and prepare a technical document or design package. A correct-looking conclusion is not sufficient if its underlying assumptions violate physical limits or safety requirements.
| Engineering discipline | Tasks |
|---|---|
| Mechanical engineering | 30 |
| Electrical engineering | 27 |
| Civil engineering | 18 |
| Structural engineering | 10 |
| Total | 85 |
Google has released 29 tasks publicly for research and development. The remaining 56 form the hidden set used for the competition leaderboard.
Human engineers helped write and validate the assignments
One of the most important details for AI Trainers Club readers is who created the benchmark.
According to the research announcement, practicing engineers supplied by Mercor helped develop and check the tasks. Of the 85 assignments, 44 were authored end to end by experts, including the project materials and grading criteria.
For the other 41 co-authored tasks, engineers first defined the problem. An AI-assisted process assembled a realistic working environment and proposed grading tools. Engineers then checked the results and strengthened the tasks.
That distinction matters. The work is not merely labeling whether an answer sounds good. It involves deciding what a technically valid result looks like, which constraints are non-negotiable, and how to test the final output.
We previously explored this kind of contribution in our article on why AI companies hire domain experts. PLE provides a recent, concrete example of that expertise being used to evaluate AI systems.
What the first results show
Google evaluated 14 frontier models across seven agent frameworks, sometimes called harnesses, which manage how models use tools and work through multi-step assignments.
Among the model-and-harness combinations highlighted in the initial hidden-test results:
| Model and agent setup | Hidden-test mean score |
|---|---|
| Claude Opus 5.5 with Claude Code | 48.3% |
| GPT 6 Astra with Codex | 45.8% |
| Claude Sonnet 5.5 with Claude Code | 43.5% |
| Gemini 4 Argon with Antigravity | 40.6% |
These are benchmark scores, not the percentage of engineering projects that could safely be completed without human involvement. A different selection of assignments, tools, or grading rules could produce different results.
There is also a potentially confusing difference in the published figures. The research paper reports a 50.6% mean for the strongest configuration across all 85 tasks. The headline 48.3% comes from the separate 56-task hidden test set. Both figures describe the benchmark, but they measure different task subsets.
Why safety checks changed the picture
Engineering is unusually unforgiving of small mistakes. A design could satisfy most requirements yet still fail because it overlooks one critical safety or feasibility condition.
Google reported that, on the co-authored tasks, the top-performing agent satisfied about 91% of individual constraints. Yet only 41.3% of its deliverables cleared every hard safety and feasibility check.
That gap is arguably more revealing than the leaderboard. Passing most checks is not the same as producing a design a licensed reviewer can accept. The benchmark rewards cost efficiency only after hard safety requirements are met.
It also shows why evaluation is moving beyond simple question-and-answer tests. Our earlier article on the race to benchmark AI agents explains why long workflows and verifiable outcomes create a very different testing challenge.
Does this create jobs for engineers?
The announcement is relevant to professional AI training work, but one distinction must remain clear: Google and Mercor did not announce a new hiring drive, a pay rate, or a guaranteed pool of engineering contracts as part of the benchmark release.
What it does demonstrate is the kind of specialized work behind advanced evaluations. Qualified engineers can help define realistic project scenarios, validate reference solutions, develop technical grading rules, and review whether AI-generated designs satisfy professional standards.
Some of these activities overlap with the broader field of AI agent evaluation and training environments. The expertise involved may be more demanding than general-purpose response ranking, especially when licenses, technical standards, or safety-critical judgments are required.
Engineers interested in this area can explore expert contracting opportunities through Mercor or micro1. These are general platform referrals, not application links for PLE. Country eligibility, screening, and available assignments vary.
Our Mercor review explains the application process and why passing an assessment does not guarantee a paid project.
What happens next
Google has opened the benchmark to researchers and developers, with 29 public assignments and an ongoing competition evaluated against the hidden set. The results may change as models, tools, and task-solving methods improve.
For now, PLE delivers a useful reality check. AI agents can work through engineering documents and generate substantial outputs, but reliably producing a safe, compliant design remains a more demanding test than answering textbook questions.
It also makes the human contribution unusually visible. Before an AI system can be judged on real engineering work, somebody with real engineering expertise has to define the work and establish what counts as correct.
Research checked November 6, 2026. Figures reflect Google’s and Mercor’s November 3 announcement and the accompanying research paper. Benchmark results may change as new submissions are evaluated.
Disclosure: This article contains referral links. I may earn a commission when eligible applicants complete qualifying activities, at no additional cost to them. Referral links do not guarantee acceptance, paid work, or earnings.
Leave a Reply