Scope of This Article (Please Read First) This article focuses on a setup where general-purpose multimodal LLMs (ChatGPT, Claude, etc.) are fed weather-chart PDFs and images drawn for human readers. This is technically a separate layer from dedicated AI numerical weather prediction models like GraphCast and AIFS. The latter never "read" a weather-chart image at all — they work directly with numerical tensors such as ERA5 data, learn the underlying physics, and have already reached the point of outperforming conventional methods on typhoon track forecasting. The "limitations" discussed here are specific to the former (LLMs reading images), not to AI meteorology in general. This distinction is the crux of the whole series. The follow-up (Part 4) will explore how to connect the "co-pilot" system designed here not to weather-chart images, but to the structured output of AI numerical forecasts. This article lays the groundwork by designing a role-sharing approach that can still deliver value even under the assumption of "reading images."

Starting Point: The Decision Not to "Close the Loop"

What became clear through Part 2 is that the structural limitations current generative AI has with (rendered) weather-chart analysis cannot be overcome by prompt engineering alone.

From here, there are two options.

  1. Don't use AI — reliable, but throws away the unique value AI offers (24/7 coverage × multiple locations × natural-language explanations)
  2. Stop trying to make AI play "the perfect forecaster" — accept the limitations and create value through a human–AI division of labor

This article takes approach 2, thinking through how to integrate AI into a real service (let's imagine a "weather explanation app").

Design Philosophy: AI as "Navigator," Human as "Pilot" — But Who's Actually the Captain?

Rather than delegating weather-chart analysis entirely to AI, we redefine the relationship as:

  • AI: extracts data and generates guidance on "what to look at"
  • Human: makes the final call

Think of it like an aircraft co-pilot. Critical flight decisions are made by the captain (human); the co-pilot (AI) reads out the instruments and checklists to verify.

The first draft of this article said that "even if AI misreads something, it won't be fatal because the user ultimately checks the weather chart themselves." The revised version directly corrects that assumption.

Correction: The co-pilot metaphor has a blind spot the first draft missed. A co-pilot's read-outs are trustworthy because the captain is experienced enough to catch errors. But the intended users of this service are hikers and members of the general public with disaster-preparedness needs — not meteorological experts. Dr. Tsuyoshi Sekiyama of the Meteorological Research Institute has stated explicitly: "Hallucinations will always occur. A layperson won't notice. Only a trained expert can sense that something is off." In other words, as long as a general user is placed in the "captain" role, there is no one on board who can detect the co-pilot's (AI's) lies.

A design that relies on "the user checks the weather chart themselves" as a safety net therefore doesn't hold up. The safety net needs to be placed somewhere else entirely.

Repositioning the Safety Net: Not Visual Verification, but "Directing Users to Authoritative Text"

We shift the final point of verification away from the user's own visual judgment and toward clear direction to official announcements and confirmed text from the Japan Meteorological Agency (JMA).

  • AI explanations are strictly for "deepening understanding" and "guiding where to look."
  • Conclusions relevant to disaster preparedness or action decisions are backed not by AI commentary, but by a clear pathway to JMA's confirmed information (advisories, warnings, short-range forecast discussion documents, point forecasts).
  • The UI provides one-tap access from each part of the explanation to the corresponding official information.

Let me be honest about the dilemma here. Claiming "AI's natural-language explanations have value" while saying "please check the official announcement" and pointing users elsewhere at the crucial moment is, in a sense, a partial self-negation of that value. This is an irresolvable tension. This design resolves it into something operationally workable by deliberately separating "using explanations to deepen user understanding and engagement" from "deferring serious decisions to confirmed information." Not pushing the role of final verifier onto the general user's eyes — that is the biggest policy shift from the first draft.

Three Roles to Assign to AI

Role 1: Highlights Guide (Directing Attention)

When a user opens the weather chart viewer, AI explains "today's key points."

Example: "Today's focus is the 850 hPa chart FXJP854. Watch how the 306K line advances northward from west of Kyushu — this is the energy source for tomorrow's heavy rain. Try moving the slider 24 hours forward to see how the warm air mass moves in."

By following the AI's guidance and interacting with the charts themselves, users get the experience of "discovering" how weather phenomena evolve on their own. What AI provides here is not "the answer" but "a perspective." The AI's role is deliberately confined to a domain where being wrong doesn't directly translate into action decisions.

Role 2: Explaining the Physical Background

For extracted data, AI supplements the "why does this happen" meteorological context.

Example: "Cold air at -30°C is present at 500 hPa (about 18,000 ft), while the surface is under a high-pressure system with clear skies. This 'temperature gap between upper and lower levels' triggers afternoon thunderstorms (atmospheric instability). Areas overlapping with the 700 hPa updraft zone deserve particular attention."

This has the potential to deliver the kind of value you'd find in a veteran forecaster's commentary — applied to every day's forecast. However, as discussed below, this only works on the condition that the underlying numbers in these explanations are fed from confirmed data, not read by AI from images.

Role 3: Cross-Checking Against External Text Data

If image analysis is weak, reinforce it with text data. Separately retrieve JMA's "short-range forecast discussion" documents and point forecast numerical data, and pass them to AI as text.

This enables something like:

"The JMA discussion document states 'as the JPCZ (Japan Sea Polar air mass Convergence Zone) shifts southward…' Checking this against the FXFE502 surface chart…"

— using text information to verify image analysis. Facts come from confirmed text; images serve as supplementary context.

Architectural Principle: "Facts Come from Structured Data"

When batch-generating weather commentary in a serverless setup (AWS Lambda, etc.), the key design considerations are as follows.

Don't Make AI Read Numerical Data from Images

Parameters like temperature, wind, and pressure by city are extracted directly from JMA's OpenData (JSON/XML) or GPV data and passed to AI as text input in the prompt.

Attempting to have AI read numbers from images carries far too high a risk of misreading, as seen in Part 2. In fact, Limitations 1 and 2 from Part 2 (projection incompatibility / inability to trace dense contour lines) stem from the fundamental input design of making a machine read images drawn for humans. Passing numbers directly sidesteps these issues at the design stage.

Pass Images Only as "Context"

Give AI the accurate numerical values extracted as text as a given, then instruct it:

"Based on these numbers, roughly read the pressure pattern behind them from the attached weather chart image, and write a highlights comment for users."

This way, AI's image analysis only handles "flavor text," and critical errors can't propagate into forecast conclusions.

Three-Tier Responsibility Structure (Revised)

The first draft's responsibility table had "final verification = user's visual check." The revised version changes where final verification is anchored.

Layer Owner Content
Data Extraction System (Lambda) Retrieves confirmed text and numerical data from JMA OpenData/GPV
Interpretation & Explanation AI (co-pilot) Adds meteorological narrative to data / focuses on directing attention and explaining background
Physical Ground Truth Confirmed data (numbers) + human experts Accuracy of the numbers underlying explanations is guaranteed by confirmed data; final responsibility for physical consistency rests with experts
Verification for Critical Decisions Pathway to JMA official announcements Disaster preparedness and action decisions are confirmed via authoritative information, not the user's own visual reading

The key point is that the final verification previously collapsed into a single "human (user)" row in the first draft has been broken out into three parts: "confirmed data," "experts," and "pathway to official announcements." The principle of not expecting general users to stand in for experts is now embedded in the architecture.

Improved Prompt: Contradiction-Detecting, Cautious Mode (with Role Clarification)

Here is an excerpt of the prompt design for running AI as a "co-pilot."

[System Prompt]
You are a "cautious data validator" with knowledge of dynamic meteorology.
Be aware that current AI vision models have serious limitations in recognizing
spatial distortions in polar stereographic projections and in tracing
densely packed contour lines.
Fabricating guesses or "plausible forecast scenarios" is strictly prohibited.

[Analysis Rules: Adhere to the following constraints absolutely]
1. Declare unreadability: If lines are crossing or densely packed and you cannot
   be confident, never guess — output "Unreadable" instead.
2. No geographic guessing: Do not infer positions from the curvature of
   latitude/longitude lines. Use only "absolute coordinates" such as clearly
   visible Japanese coastlines as reference points.
3. Eliminate overconfidence: Avoid definitive expressions ("this will be..."
   "the forecast is...") and limit yourself to "this appears to be..."
   or "there is a possibility that..."

[Execution Steps]

■ Step 1: Self-Assessment of Visual Confidence (Metacognition)
For each attached chart, assess the "readability of lines" around the Japanese
archipelago and declare a rating of [High] / [Medium] / [Low].
If [Low], stop making inferences from that chart.

■ Step 2: Local Data Extraction (No Guessing)
Limited to the Japanese archipelago, extract only values and features that can
be clearly read from the chart, in bullet-point form.
Skip any elements that cannot be read.

■ Step 3: Physical and Climatological "Contradiction Detection" (※ as a supplementary check)
Combine the extracted data and run it through a filter for "physically
inconsistent points" against dynamic meteorology and Japanese climatology.
If a contradiction is found, flag it as "likely an image-reading error"
(do not make a final determination).

■ Step 4: Limited Weather Condition Commentary After Filtering Out Contradictions
Use only data that passed Step 3 to provide a limited commentary.

The core of this prompt is three things: "don't generate forecast scenarios," "make it admit when it can't read something," and "filter for physical contradictions."

Important role clarification: Physical contradiction detection is demoted from "lead role" to "supporting role." The first draft said to "make Step 3 contradiction detection the lead role." But since Part 2, Limitation 3 concluded that "LLMs have no physical model and do not understand fluid dynamics," having that same LLM take the lead on detecting physical contradictions is fundamentally strained. You cannot ultimately delegate physical consistency judgment to something that doesn't understand physics. Therefore, in the revised version, contradiction detection is downgraded to "a first-pass filter (supplementary) that catches obviously wrong outputs." The ultimate guarantee of physical correctness rests not on AI self-checking, but on (1) the accuracy of the confirmed data (numbers) themselves and (2) human expert judgment. Prompt-based contradiction detection remains as a preliminary cut to reduce "obvious accidents" before expert review, but no more trust than that is placed in it.

And even with all this, hallucinations won't disappear entirely. It's important to recognize that prompting is a technique for reducing the probability, not a means of overcoming the limitations themselves.

Addressing Remaining Risks: Designing Disclaimers

Even with careful prompting, hallucinations remain. The honest and responsible approach is to disclose this transparently to users rather than hiding it in the system.

This is especially essential when the intended users include hikers, aviators, and people with disaster-preparedness needs — situations where weather judgment can be a matter of life and death. And as noted above, those users are not experts who can spot a lie — which is precisely why disclaimers must not say "check it yourself" but instead direct users to "consult the external authoritative source: confirmed information."

Pattern A: Everyday Display (Inline)

A lightweight disclaimer shown persistently just below the explanation text.

⚠️ AI-generated reference information This commentary reflects AI's independent reading of weather chart trends. It may contain misidentifications of locations or values (hallucinations) when analyzing complex charts. Such errors are difficult to notice without expert knowledge. For final weather judgments and disaster preparedness decisions, please do not rely on your own reading — always check JMA's official announcements.

Pattern B: Onboarding (Terms of Use)

A somewhat more formal risk disclosure presented at the start of use.

About the AI Weather Chart Explanation Feature This feature is designed to support interpretation of weather models and upper-air charts and to deepen meteorological understanding. Due to the nature of generative AI, inaccurate or internally inconsistent information may be produced when extracting spatial information or numerical values from images (weather charts). Such errors are presented in natural, confident-sounding language, making them difficult for non-experts to detect. Please do not use this AI commentary as your sole basis for critical decisions involving human life or property, including mountaineering, maritime navigation, aviation, agriculture, and disaster preparedness during typhoons or heavy rain. Always refer to information published by official agencies such as the Japan Meteorological Agency for accuracy.

Pattern C: High-Risk Situations Only (Dynamic Alert)

Displayed in a prominent color only when a typhoon is approaching or a special weather warning is in effect.

🚨 [Important] Using This Service During Severe Weather Current weather conditions present a risk of serious meteorological disaster. AI analysis contains errors that cannot be identified by appearance alone. Using this commentary as a basis for disaster preparedness decisions is extremely dangerous. Please immediately check JMA's latest information and your local government's evacuation guidance.

"Report an Error" Button in the UI

In addition to disclaimers, it is recommended to include a "Report an AI Error" button (e.g., a thumbs-down icon).

This carries significance beyond a simple feedback mechanism.

  • It serves as evidence that users are engaging critically (liability mitigation)
  • It generates test data for future prompt improvements
  • It acts as a UX signal that discourages over-reliance on AI

To be honest, the error-report button has its own limitations. Lay users can't report errors they don't notice in the first place. This button can only catch "the subset of errors someone happened to spot." That's exactly why the report UI is a supplement, not the main safety net. The main safety net, as I keep saying, is the pathway to confirmed information.

Summary: Design the "How to Use It," Not Just the "Correctness"

Across three parts, we've examined the possibilities and limitations of generative AI for (rendered) weather-chart analysis. Here's the final summary.

What AI can currently handle:

  • Data extraction from text information and color charts (to a limited degree)
  • Building explanations based on existing text data
  • Creating checklists of "what to look at next"
  • First-pass filtering of physical contradictions (supplementary — not a final guarantee)

What AI currently cannot handle:

  • Precise reading of black-and-white, high-density upper-air charts
  • Building a coherent three-dimensional structure across multiple chart levels
  • Final climatological and physical judgment
  • Honest self-awareness of "what it can't read" (can be reinforced by prompting, but not perfectly)
  • And: offloading the role of catching lies onto non-expert users

If you're aiming for practical deployment despite all this, the three pillars of the design are:

  1. Ground truth of data is guaranteed by structured text, not by AI
  2. AI acts as "co-pilot," focusing strictly on directing attention and explaining background
  3. Limitations are disclosed transparently through disclaimers and an error-report UI, and critical decisions are caught by a clear pathway to confirmed information (JMA official sources)

The biggest difference from the first draft is point 3. Rather than entrusting final verification to "the user's eyes," entrust it to "the external authoritative source: confirmed information." Given that the intended users are not experts, this is what I believe to be the honest and responsible landing point.

Not "waiting for AI to become perfect," but "designing a division of labor where value emerges even with imperfect AI" — this is what I consider the most realistic and honest approach available right now.

Afterword: What I Really Wanted to Convey with This Series

When using AI in a specialized domain, the thing to be most wary of is not "AI making mistakes."

It's "AI making mistakes in a confident, plausible-sounding tone."

The output observed in Part 1 — "I have verified that everything is fully consistent" — would have circulated as "correct commentary" if the user hadn't noticed the logical contradiction. And only a trained expert would notice.

When integrating AI into professional work or personal hobbies, two wheels are indispensable: a prompt design that makes AI articulate its own limitations, and a workflow design that delegates final correctness to humans (ideally experts) and confirmed information. I hope this series serves as a useful reference when thinking through that kind of design.