Google’s MKA Benchmark Tests AI Agents on 200 Visual Knowledge Tasks
A 200-task evaluation of AI agents working with diagrams, spreadsheets and other complex files reveals a more demanding way to measure knowledge work.
Original conceptual illustration generated for DeepTech; not a photograph or image of the reported experiment.
Can an AI agent read an engineering drawing, inspect a spreadsheet and produce a usable deliverable? That question is harder to answer than whether a model can summarize text. On 10 November 2026, Google DeepMind and Artificial Analysis introduced the Multimodal Knowledge Agent (MKA) benchmark to evaluate agents working with visually complex professional files. The benchmark has 200 private tasks across 10 industry verticals. It offers a useful framework for enterprise buyers, but it is not proof that an agent can reliably replace a specialist.
The important distinction is between completing an artificial test and operating inside an organization with real permissions, incomplete records and consequences for mistakes. MKA makes the former more representative of actual deliverables than many conventional question-answer benchmarks; the latter still requires separate validation.
What exactly does MKA evaluate?
According to Google’s launch explanation, subject-matter specialists helped design a set spanning 83 occupations. The tasks involve input bundles such as plans, diagrams, tables and document layouts. Instead of answering a multiple-choice question, an agent may need to prepare a slide deck, structured report, spreadsheet or marked-up drawing.
The stated human work involved in a task is typically three to eight hours. That describes task construction and difficulty; it does not establish that a model saves those hours in a workplace. Output correctness, quality control and human editing must still be measured independently.
A scoring system that looks inside deliverable files
Each task comes with an expert-written rubric of roughly 70 binary checks. Google says the rubric places 75% of its weight on factual accuracy and 25% on visual design. Artificial Analysis describes its headline metric, Climb Weighted Satisfaction Rate (Climb WSR), as the average weighted share of hard rubric criteria met across tasks, rather than a general probability that a project is correct.
The evaluation uses the open-source Stirrup agent harness. Models receive the same basic tooling and a 250-turn budget. Artificial Analysis uses an agentic judge to open submitted artifacts, render them and apply the rubric. Google reports 96% agreement with domain experts in a comparison of judgment results, but that statistic alone does not prove all grading decisions are accurate or that every kind of failure is captured.
What the published leaderboard actually says
The Artificial Analysis MKA page lists Claude Fable 5.2 at 57% Climb WSR, followed by Gemini 4 Carbon at 54% and Claude Opus 5.5 at 53% in the configurations shown when the announcement was published. These are benchmark-specific results, sensitive to model version, configuration, prompting, tools, scoring changes and cost accounting. They are not percentages of correct professional decisions in the field.
Google separately says Gemini 4 Carbon reaches second place on quality while costing about six times less per task than the benchmark leader. That is a claim about the chosen evaluation’s token-related cost measure; it excludes many real enterprise expenses such as integration, operational monitoring, security reviews, document preparation and human verification. Procurement comparisons should record the date and configuration of any leaderboard snapshot.
The difference between independent execution and independent test design
Artificial Analysis states that it executes the model evaluations independently. However, Google and Artificial Analysis collaborated on the benchmark itself. Those facts can coexist. Independence of execution does not mean a benchmark was created without stakeholder involvement, nor does it invalidate the published methods. For an enterprise buyer, transparent governance, hidden holdout tasks and disclosure of the evaluation setup matter more than promotional slogans.
The full 200-task set is private to reduce test-data contamination. A separate set of 10 examples is public through Hugging Face. Public examples improve inspectability, but their small size does not independently reproduce the private benchmark’s published rankings.
Where the evidence is useful for business buyers
Visual file handling is central to construction, engineering, insurance, scientific research and software development. A sensible enterprise evaluation would start by identifying one type of deliverable the organization already produces, then obtain permission to test representative files. The pilot should check not only whether the agent produces a document, but whether it cites the correct drawing revision, preserves numeric units, handles missing pages and clearly identifies uncertainty.
- Accuracy: independent checks of extracted values, cross-file references, units and source provenance
- Artifact quality: spreadsheet formulas, page layout, editable objects and missing sections
- Workflow reliability: repeat runs, failure recovery, tool availability and human rework
- Privacy: treatment of sensitive files, retention, access controls and exported telemetry
- Total cost: inference plus preparation, oversight, corrections and system integration
What MKA does not establish
A high benchmark result is not evidence of safety in a financial, healthcare or industrial deployment. MKA does not by itself measure whether an agent obeys an organization’s permission model, resists prompt injection, meets a service-level agreement or performs on unfamiliar proprietary documents. A benchmark can inform model selection; it cannot replace controlled field trials and independent acceptance criteria.
For the operational questions behind those trials, read our Enterprise AI Agents deployment checklist and our guide to independent AI validation. Both focus on observed outcomes and limits rather than a headline accuracy number.
Editorial assessment
MKA makes a credible methodological contribution by testing completed multimodal artifacts under documented conditions. It provides a useful comparison of particular model configurations, not a universal ranking of business readiness. DeepTech has not rerun the private evaluation, audited the graders or conducted an independent deployment test. Any interpretation here rests on the publicly described protocol and published results as of 11 November 2026.
