Starting Point: The Decision to "Not Wrap It Up Neatly"

What became clear through Part 2 is that the structural limitations current generative AI has in weather chart analysis cannot be overcome by prompt engineering alone.

From here, there are two options.

  1. Don't use AI at all — reliable, but you throw away the unique value AI offers (24/7 coverage × multiple locations × natural-language explanations)
  2. Switch to a design that doesn't ask AI to "play the perfect forecaster" — accept the limitations upfront and create value through a human-AI division of labor

This article explores option 2: designing a real service (let's imagine a "weather commentary app") that incorporates AI on those terms.

Design Philosophy: AI as "Navigator," Human as "Pilot"

Rather than delegating weather chart analysis entirely to AI, we redefine the relationship as:

  • AI: extracts data and generates guidance on "what to look at"
  • Human: looks at the actual weather chart with their own eyes and makes the final call

Think of it like an aircraft's first officer. The captain (human) makes the critical flight decisions; the first officer (AI) reads out the instruments and checklists to verify.

With this mindset, even if AI misreads something, it's not fatal — because the user ultimately looks at the weather chart and confirms it themselves.

Three Roles to Delegate to AI

Role 1: Highlight Guide (Directing Attention)

When a user opens the weather chart viewer, AI explains "today's key points to watch."

Example: "Today's focus is the 850 hPa FXJP854 chart. Watch how the 306K line moves northward from west of Kyushu — this is the energy source for tomorrow's heavy rain. Try sliding the time slider 24 hours forward to see how the warm air mass moves in."

By following the AI's guidance to interact with the charts, users get the experience of "discovering" meteorological changes for themselves.

Role 2: Explaining the Physical Background

For extracted data, AI fills in the meteorological "why" — the physical reasoning behind what's happening.

Example: "Cold air at -30°C is moving in at 500 hPa (about 18,000 ft), but the surface is under a high-pressure system with clear skies. This 'temperature gap between upper and lower levels' is what triggers afternoon thunderstorms (atmospheric instability). Areas that overlap with the 700 hPa updraft zone need particular attention."

This has the potential to deliver the kind of value you'd find in a veteran forecaster's commentary — every single day.

Role 3: Cross-Checking Against External Text Data

If image analysis is weak, reinforce it with text data. Separately retrieve the Japan Meteorological Agency's "Short-Range Forecast Discussion" and numerical point-forecast data, then pass them to AI as text.

This enables a flow like:

"The JMA discussion states 'as the JPCZ (Japan Sea Polar air mass Convergence Zone) moves southward…' Cross-referencing this against the FXFE502 surface chart…"

— where text information is used to fact-check the image analysis. Ground truth comes from confirmed text; images serve as supplementary context.

Architectural Insight: "Facts Come from Structured Data"

When batch-generating weather commentary in a serverless setup (AWS Lambda, etc.), here are the key design principles.

Don't Make AI Read Numerical Data from Images

Parameters like temperature, wind, and pressure for each city should be extracted directly from JMA's OpenData (JSON/XML) or GPV data and passed to AI as text input in the prompt.

As we saw in Part 2, trying to have AI read numbers from images carries far too high a risk of misreading.

Pass Images Only as "Context"

Give AI the accurate numerical values extracted from text as a given, then instruct it:

"Based on these numbers, roughly read the pressure pattern from the attached weather chart image and write a highlight comment for the user."

This way, AI's image analysis only handles the "flavor" of the narrative — critical errors can't cascade into the forecast conclusion.

A Three-Layer Division of Responsibility

Layer Owner Content
Data Extraction System (Lambda) Retrieves confirmed text and numerical data
Interpretation & Commentary AI Adds a meteorological narrative to the data
Final Verification Human (user) Confirms with their own eyes in the chart viewer

Improved Prompt: Contradiction-Detection, Cautious Mode

Here is an excerpt of the prompt design for running AI as a "first officer."

[System Prompt]
You are a "cautious data validator" with expertise in dynamic meteorology.
Be aware that current AI vision models have serious limitations in recognizing
the spatial distortions of polar stereographic projections and in tracing
densely packed isolines.
Fabricating speculative "plausible forecast scenarios" is strictly prohibited.

[Analysis Rules: Adhere to the following constraints absolutely]
1. Declare unreadability: If lines are crossing or densely packed and you cannot
   be confident, do NOT speculate — output "Unreadable" instead.
2. No geographic guessing: Do not infer positions from the curvature of latitude/
   longitude lines. Use only "absolute anchors" such as clearly visible Japanese
   coastlines.
3. Eliminate overconfidence: Avoid assertive language ("this will be," "the
   forecast is"). Limit yourself to "appears to be," "there is a possibility of."

[Execution Steps]

■ Step 1: Self-Assessment of Visual Confidence (Metacognition)
For each attached chart, assess the "readability of lines" around the Japanese
archipelago and declare one of three levels: [High] / [Medium] / [Low].
If [Low], stop making inferences from that chart.

■ Step 2: Local Data Extraction (No Speculation)
Restricted to the Japanese archipelago, extract only values and features you can
clearly read from the chart as a bullet list. Skip anything you cannot read.

■ Step 3: Physical/Climatological Contradiction Detection
Combine the extracted data and check for anything "physically inconsistent"
against dynamic meteorology and Japanese climatology.
If a contradiction is found, conclude: "This is likely an image-reading error."

■ Step 4: Limited Weather Commentary After Eliminating Contradictions
Use only data that passed Step 3 to provide a limited commentary.

The core of this prompt is three things: "don't let it construct forecast scenarios," "make it admit when it can't read something," and "make contradiction detection the star of the show."

That said, this doesn't eliminate hallucinations entirely. It's important to recognize that prompts are a tool for reducing the probability — not a way to overcome the underlying limitations themselves.

Handling Residual Risk: Designing Disclaimers

Even with careful prompting, hallucinations remain. The honest thing to do is not hide this from users, but disclose it transparently at the system level.

This is especially non-negotiable if you're designing for users where weather judgment can be a matter of life and death — hikers, pilots, disaster preparedness, and so on.

Pattern A: Everyday Display (Inline)

A lightweight notice shown persistently just below the commentary text.

⚠️ AI-generated reference information This commentary reflects the AI's independent interpretation of weather chart trends. When analyzing complex charts, it may contain misidentifications of positions or values (hallucinations). For any final weather judgment or disaster-preparedness action, always check the Japan Meteorological Agency's official announcements.

Pattern B: Onboarding (Terms of Use)

A slightly more formal risk disclosure presented at first use.

About the AI Weather Chart Commentary Feature This feature is intended to support the interpretation of weather models and upper-air charts to deepen your understanding of meteorology. Due to the nature of generative AI, inaccurate or contradictory information may be produced when spatially recognizing images (weather charts) or extracting numerical values. Do not rely solely on this AI's commentary for critical decisions involving human life or property — including mountaineering, maritime navigation, aviation, agriculture, and disaster preparedness during typhoons or heavy rain. Always refer to information published by official agencies such as the Japan Meteorological Agency for accuracy.

Pattern C: High-Risk Situations Only (Dynamic Alert)

Inserted in a prominent color only when a typhoon is approaching or a special weather warning is in effect.

🚨 [Important] Using This Service During Severe Weather Current conditions indicate a risk of serious weather-related disaster. Because AI analysis contains errors, using this commentary as the basis for disaster-preparedness decisions is extremely dangerous. Please check the JMA's latest information and your local government's evacuation advisories immediately.

"Report an Error" Button in the UI

In addition to disclaimers, it's recommended to add a "Report an AI Error" button (a thumbs-down icon, for example).

This carries meaning beyond a simple feedback feature.

  • It becomes evidence that users are engaging critically (liability protection effect)
  • It generates test data for future prompt improvements
  • It serves as a UX signal that discourages over-reliance on AI

Wrap-Up: Design the "How to Use It," Not Just the "Correctness"

Across three parts, we've examined the possibilities and limitations of generative AI in weather chart analysis. Here's the final summary.

What AI can handle right now:

  • Data extraction from text information and color charts (to a limited extent)
  • Building commentary based on existing text data
  • Turning "what to check next" into a checklist
  • Detecting physical contradictions (depending on the prompt)

What AI cannot handle right now:

  • Precise reading of black-and-white, high-density upper-air charts
  • Coherently constructing 3D atmospheric structure across multiple chart levels
  • Final climatological or physical judgment calls
  • Honestly recognizing "I can't read this" (can be reinforced with prompts, but not solved)

If you're aiming for practical deployment despite all this, the three pillars of design are:

  1. Ground truth in data comes from structured text, not from AI
  2. AI acts strictly as "first officer" — directing attention and explaining background
  3. Don't hide the limitations — disclose them transparently through disclaimers and an error-reporting UI

Rather than "waiting for AI to become perfect," the goal is to "design a division of labor where value emerges even with an imperfect AI." That, I believe, is the most realistic and honest approach available right now.

Afterword: What I Really Wanted to Say with This Series

When using AI in a specialized domain, the thing to be most wary of is not "AI making mistakes."

It's "AI making mistakes confidently, in a tone that sounds completely plausible."

The output we observed in Part 1 — "I have verified that everything is fully consistent" — would have circulated as "correct commentary" if the user hadn't noticed the logical contradiction.

When integrating AI into professional or hobby workflows, you need both wheels turning: a prompt design that gets AI to articulate its own limitations, and a workflow design where a human does the final check. I hope this series serves as a useful reference when thinking through that kind of design.

End of Series