Monday, December 17, 2007

Communicating Uncertainty to the Public

People want answers. How much is it going to snow in Orchard Park? Could the sleet make it up to Wheatfield by 9am tomorrow? Those kinds of answers. Sometimes we have those answers but many times we do not.

To start with, meteorology is a physical science which is--to say the least--inexact. We are on a spinning globe, 3/4 of which is covered in water and ice, being heated by a thermonuclear furnace 93 million miles away. We're dealing with fluids and gases and how they interact, along with differing land topography, plant cover, snow cover, & different reflectivity of the land (dark soil absorbs far more of the sun's heat, while ice and snow reflect much of it back toward space), to name a few of the changing variables. Sea surface temperatures, and where warmer or colder temperatures are located in the Pacific have a HUGE effect on weather as well. So, weather is not something akin to Leggo blocks where everything snaps neatly into place.

As for the issue of uncertainty, not all forecasts are created equally. I'm not referring to the training and experience of various Buffalo tv weathercasters and meteorologists--I'm referring to forecast confidence. Verification scores of forecasts generally show that competent meteorologist produce forecasts that are rather accurate 80% of the time for the first 36 hours. But that includes some relatively easy calls, such as when we're in a stagnant, stable weather pattern in which conditions will vary little for a few days. For the tougher calls, I'd venture a guess those probably register closer to 60% accuracy, which would still be pretty good, and getting better.

When meteorologists occur scientific conferences (for example, WIVB sends me to an American Meteorological Society once each year for continuing education), a great deal of time is spent by lecturing scientists on the necessity to communicate UNcertainty, in order that our public understands a particular forecast carries certain probabilities of not working out. The NWS uses probability percentages of precipitation as one way of communicating uncertainty. For myself, I've chosen not to use them in our on-air forecasts because my experience tells me many people don't understand the meaning of those percentages. A 30% chance of snow, for example, seems to mean to many that 30% of the area will have snow. That's not what it means, though. A 30% probability means that any given location in a forecast area has a 3 in 10 chance of having measurable snow. I've even seen a few meteorologists make this error. So, I use more informal means of communicating uncertainty, as you've seen on this blog prior to the weekend storm and on the air--letting folks know that computer models are at odds with one another. Sometimes it's in my inflection pattern and, I suppose, facial expression. If I had a paying client who wanted the most precise information possible, I would assign probability percentages to that forecast.

In any case, it's very important for you to know the uncertainty in a forecast or, for that matter, in environmental concerns and scientific controversies. Let's take it to a level where the stakes are higher; suppose you go to your internist with a worsening dull ache in your abdomen. After ordering tests, I think you'd want your doctor to give you the complete perspective on how certain or uncertain he or she is about your diagnosis before he begins treatment.

So, we're not going to present each and every forecast to you as if they were all created equally. Some are high confidence forecasts, and some are low, and you need to know that. Speaking of which, as of today, a forecast for a White Christmas with new snow is a lower confidence forecast.

19 Comments:

Blogger Ayuud said...

1st Comment and yep thats the truth so people dont whine about the forecast or snow amounts in my opinion if i were metroligist i would say "god knows how much snow will get or the forecast"



hahahahahahahahah

December 17, 2007 2:57 PM  
Anonymous what about op? said...

Sorry I kept asking about Orchard Park, I didn't know how uncertain you guys were. You DID do a good job, by the way, though, because we did end up getting around ten inches of snow, exactly what you told me, so thanks :)

December 17, 2007 4:42 PM  
Anonymous citygirl said...

hows the metro area going to look tonight?

December 17, 2007 8:24 PM  
Blogger Mary Beth Wrobel said...

I must say it's tougher to predict narrowing town to town accumulations, especially when dealing with a large-scale storm as was the case this weekend. This storm involved a double-barreled Low and had a large scale impact from New England, up and down the east coast and back through the Great Lakes and portions of the Midwest. That's a lot of turf spread across over 1000 miles! Lake effect is a completely different animal. It's on a much smaller scale, and wind direction is key among other atmospheric criteria in deciding if, say for example, Orchard Park gets snow (and how much), or if the sun will be shining. It's only natural that we've gotten comfortably accustomed to getting in touch with "how much snow is our town going to get versus another?" because of our unique Lake Effect phenomenon.

December 17, 2007 8:54 PM  
Anonymous Blizzard said...

Big Low looks to be passing to our west on 12/24 and setting up shop above us..Could this be true..
SW winds???? LES??
MB/DON, what do you think?
http://www.weather.unisys.com/gfsx/7e/gfsx_500p_7e.html

December 17, 2007 9:23 PM  
Blogger Mary Beth Wrobel said...

Blizzard: Here's my thoughts on that...way to early to tell. Plus, lake effect is small scale stuff.

December 17, 2007 11:25 PM  
Anonymous Lakeshadow said...

ok...heres a website with a great breakdown of what it takes to become a successful forecaster and some great forecasting philosophy. I would be very interested what you mets thought of this:
http://www.theweatherprediction.com/philosophy/forecaster/

December 18, 2007 11:35 AM  
Anonymous lakeshadow said...

the end of that url:
asting/

December 18, 2007 11:37 AM  
Anonymous WeatherMom said...

In physics, the more things interacting, the greater the number of possible outcomes. I find it amazing that today's meteorologists can do as well as they do. This past weekend was quite a challenge to predict. Thanks for sharing the problem-solving process with us.

December 18, 2007 12:07 PM  
Blogger Don Paul said...

Jeff Haby's site is fairly well thought of by broadcast meteorologists. I recommend starting at the home page and taking it one step at a time.

But you have to remember that much of what's there is written on the premise that the reader has a good, fundamental education in meteorology. So, if you come to some elements which seem especially tough to decipher (and Jeff writes clearly), don't get too discouraged.

December 18, 2007 1:57 PM  
Anonymous Anonymous said...

After reading your blog on the uncertainty of forecasting, and hearing Regis Philbin say, on his show, all weathermen/women should be fired for crying wolf over the last storm, I'm still on the fence about the percentages. With all the technology in the last 20 years, one would think forecasting should be that much easier. But after reading your post, it doesn't sound that easy. I know you hate to downplay a storm and then have it blow up in your face, then again if it gets out of control, you're the bad guy. It's a tough call. Do you think all the technology helps forecasting or is it that much more complicated because of all the information you have to sift through?

December 18, 2007 6:19 PM  
Blogger Mary Beth Wrobel said...

Lakeshadow: I think J. Haby's tips are terrific. He makes some excellent points. The reason why he says getting a met degree doesn't guarantee you'll be a good forecaster is because weather forecasting is not just science, it's also an art. It definitely takes skill and experience and being in tune with the local climatological variables that are unique to individual cities, i.e. changing topography, proximity to lakes, or even bigger bodies of water, or even a lack of water such as in the Midwestern states which can have a huge impact on local weather forecast verification. I do believe it's critical to have a solid understanding of the math and physics that go into making the models/weather data which we utilize to create forecasts. Oh, one thing that he didn't make mention of when forecasting...Look out the window! Ha!...But oh so true!

December 18, 2007 6:20 PM  
Anonymous LakeShadow said...

Thanks! I'm glad to hear Jeff Haby a good resource. I've spent all day reading and studying his page, now my brain is swimming...but its great! I'm already starting to understand a bit more as I go.
He does recommend a good Meteorology textbook....have any in mind that would be good?
Thanks again!

December 18, 2007 7:17 PM  
Blogger Don Paul said...

To anonymous, on the question of technology. The additional technology brings many benefits to meteorologists but in terms of workload, it has its disadvantages.
For one thing, there are so many layers to the forecasting onion now, it sometimes feels like we're getting information overload, both in the private sector and at the NWS. We have tools we couldn't have imagined existing back in my college days, but it's hard to always find the time to use all of them.

Fortunately, there is continuing education available for meteorologists online through a wonderful program with the acronym COMET, in which we can take and be examined in short courses in interpretation of Doppler radar datasets, hydrology, snow forecasting, lake effect, severe convection, and the list goes on and on.

In the end, we are much better off for having these new technologies. But because many applications are now run at our fingertips, I find (as do many of my broadcast and NWS colleagues) my math and physics skills in dealing with equations have eroded, because the computers do the calculus and work the physics equations for us now. And, I suppose there was a certain fatalism in an earlier era, as recent as the early 80s, when data was limited to the point where you did your analysis, you knew there were holes in the data no one could fill, and you took your best scientific shot. Now, it often seems there's never enough time to get through all the data we have. But even so, there is still an incomplete understanding of how the atmosphere is going to behave (Mike Cejka alluded to this in an earlier post on another thread). Every single thing that happens in the atmosphere is controlled by the laws of physics, but there are so many, many variables and permutations on what may happen that the possibilities are huge in number. Computer models have been an enormous addition in organizing what might otherwise seem nearly chaotic behavior of molecules (it's NOT chaotic--it just seems that way) and extrapolating forecast scenarios from realtime data.

Forecast accuracy has been steadily increasing over recent decades. The pace of improvement seems slow to the public, but it's really been remarkable. 5 day outlooks were considered nearly worthless when I attended college, and now we're into 7 day outlooks (the 6th and 7th day don't work out as often as the 4th and 5th, of course). Severe weather forecasting has improved immensely, and no one will ever be hit by a "surprise" hurricane like the 1938 storm which killed hundreds on Long Island and parts of srn New England. Tornado warning leadtime continues to increase, especially with larger, more dangerous storms, thanks to Doppler Radar.

The latter is a particularly interesting tool. Many tv stations have purchased Doppler radars even when their staff has little training in the atmospheric sciences, let alone radar interpretation. That's because audience research shows the public wants to know the station they're watching has the right tools, and it's worth it to the station to make such a purchase. The promotion of a good Doppler radar can actually make some difference in the ratings. The irony is that most of what we show you on the air is radar reflectivity; the detection of precipitation. The Doppler-derived products, used mainly offair, are the real advance in radar. They use the motion of raindrops, snowflakes, dust, insects and just water vapor to map out what the winds are doing aloft, and whether dangerous circulations are developing within a thunderstorm cell. Some tv weathercasters have no training in the use of these products, and brag about "live doppler radar" showing you where it's raining or snowing. Well, the old conventional radars could do that, too, before the advent of Doppler radar. There's nothing "Doppler" about such presentations on the air. Once in a while, we've shown you the complex wind velocity displays from Doppler radar, but it takes a lengthy discussion to make that kind of presentation digestible to the public. One great time saver 4Warn Doppler radar has is built in hazard detection, based on what are called algorithms, to find hail, rotation, actual tornado signatures on occasion, wind shear, and track by precise speed and direction the path of these storms. Having reliable hazard detection algorithms gives us an advantage I would have found inconceivable just 15 or 20 years ago. Not foolproof, but reliable. I hope the same can be said about me!

As in any science, those who make no effort to keep up surely get left behind.

December 18, 2007 8:53 PM  
Anonymous Marshall Stack said...

Over the weekend (Sunday evening?), I think one of you guys showed a map of the wind speed and direction at various sites over WNY, and explained briefly how it could be used to track the center of rotation of the Low over our area. Was that derived from Doppler?

And here's a dumb question: There's a large white Epcot Center-looking globe on a small tower by the Thruway near the airport. Is that the Buffalo area Doppler radar?

December 19, 2007 8:20 AM  
Blogger Don Paul said...

Marshall,

Those wind directions and speeds were actually from surface observations, mainly at airports. However, Doppler radar can find such circulations as well, though not quite a the surface, since the beam is elevated at different angles.

And, yes, that "Epcot" ball is the Buffalo NWS 88-D radar. There are more than 135 of them around the nation, plus similar radars at a number of Air Force bases, and somewhat less costly radars at a number of large airports used by the FAA (called Terminal Doppler radar).

December 19, 2007 12:03 PM  
Anonymous Marshall Stack said...

Is the rotation of a High or Low easier to pinpoint using surface observations, or at higher altitude? Do you get a better idea of its location that way, or is a barometric reading easier (i.e. the center of the system being the area of lowest/highest pressure)?

Two more questions (I'm nosy!)- Can Doppler radar detect precipitation/winds/etc at different altitudes? What is the effective range of such radar, and does the curvature of the earth affect said range?

Ok, that was actually three questions. I'm on my lunch break, and these things sort of just pop in my head.

December 19, 2007 12:48 PM  
Blogger Don Paul said...

To find a surface low, it's best to use surface observations. The network of observing sites is more concentrated. For areas of low pressure aloft, soundings of the atmosphere at various levels taken by weather balloons are very useful. Circulation centers can be extrapolated from radar reflectivity and Doppler-derived products. Upper level areas of low pressure are generally displaced from the surface low, and require soundings and satellite imagery for location. When the upper low is "stacked" vertically above the surface low, that low is usually very mature, moving more slowly, and losing intensity.

Doppler radar can be elevated to many angles to examine different altitudes within the atmosphere, or a storm. The NWS 88-D Doppler performs what's called a volumetric scan of the atmosphere. The antenna starts at a low angle (say, .5 degrees above the surface) and rotates numerous times, each time with an increased angle of elevation, to completely scan a storm. This scan takes longer than a single sweep by the radar because it gathers so much more information, deducing patterns of circulation, shear, wind velocities, hail and, occasionally, tornadoes. This Doppler-derived wind data requires a range limitation to 124 nautical miles. Beyond that range, the beam is too high to find low level circulations and winds in a thunderstorm, and the Doppler data becomes less reliable. The range for reflectivity (echoes of precipitation) is about 240 nautical miles. Because of the curvature of the earth, the straight radar beam (actually many rapidfire pulses of electromagnetic energy) is very high in the atmosphere at 240 miles. At that range, the radar will overshoot all but the tops of very intense, vertically developed thunderstorms.

December 19, 2007 2:38 PM  
Anonymous Marshall Stack said...

Thanks for the info! This is great stuff! Another question: You said that a "Stacked" low is slower, and loses intensity. When you see two lows merge like in that "Perfect Storm" movie, and form a stronger system, are those surface lows, or upper-level lows?

December 21, 2007 9:36 AM  

Post a Comment

Links to this post:

Create a Link

<< Home