Meta tags:
description=2 posts published by troyca during April 2011;

Headings (most frequently used words):

april, 2011, recent, cmip3, model, troy, scratchpad, 29, 22, pages, comments, blogroll, posts, search, archive, hindcasts, and, ar, weather, noise, multi, mean, vs, individual, runs, in, 20th, century, hindcast,

Text of the page (most frequently used words):
the (172), and (41), model (30), this (27), runs (20), mean (18), for (17), not (16), multi (16), 2011 (15), that (15), models (14), mmm (14), noise (14), than (14), #individual (13), from (12), with (11), 2012 (11), observations (11), does (10), year (10), better (10), trend (10), 2014 (9), rmse (9), why (9), forced (9), what (9), 2013 (8), post (8), but (8), different (8), century (7), between (7), april (6), may (6), temperatures (6), 20th (6), case (6), well (6), ipcc (6), here (6), are (6), more (6), any (6), seem (6), might (6), weather (6), some (6), like (5), sensitivity (5), surface (5), hindcast (5), temperature (5), ensemble (5), would (5), actual (5), all (5), value (5), linear (5), average (5), one (5), each (5), way (5), look (5), comments (4), troy (4), scratchpad (4), august (4), february (4), september (4), october (4), efficacy (4), forcing (4), observed (4), another (4), recent (4), climate (4), has (4), cmip3 (4), averages (4), correlation (4), diff (4), best (4), members (4), compared (4), process (4), response (4), series (4), was (4), around (4), generally (4), these (4), using (4), run (4), there (4), should (4), following (4), perform (4), fluctuations (4), again (4), errors (4), line (4), which (4), after (4), can (4), wordpress (3), com (3), comment (3), 2010 (3), november (3), december (3), january (3), march (3), july (3), cmip5 (3), two (3), enhancement (3), kummer (3), dessler (3), warming (3), estimates (3), could (3), global (3), against (3), carbon (3), about (3), based (3), values (3), minus (3), explain (3), least (3), course (3), explanation (3), less (3), fake (3), lower (3), time (3), consider (3), along (3), simply (3), will (3), use (3), although (3), within (3), hadcrut (3), flat (3), simple (3), seems (3), see (3), hindcasts (3), considered (3), looking (3), data (3), image (3), component (3), were (3), differences (3), site (2), required (2), report (2), log (2), sign (2), subscribed (2), subscribe (2), have (2), june (2), search (2), uncertainty (2), estimating (2), example (2), processing (2), science (2), doom (2), graphs (2), day (2), lucia (2), multiple (2), reflect (2), environmental (2), phase (2), relation (2), atmospheric (2), dioxide (2), just (2), properties (2), outperforms (2), still (2), deviation (2), itself (2), red (2), somewhat (2), showing (2), had (2), worse (2), both (2), first (2), then (2), result (2), other (2), getting (2), out (2), most (2), pretty (2), being (2), their (2), scatter (2), correct (2), closest (2), results (2), chapter (2), appears (2), particularly (2), given (2), difference (2), metric (2), performs (2), once (2), only (2), refers (2), 1900 (2), 1950 (2), those (2), shows (2), basically (2), figure (2), error (2), while (2), independent (2), high (2), score (2), also (2), question (2), chance (2), got (2), code (2), future (2), last (2), ar4 (2), filed (2), under (2), troyca (2), uncategorized (2), bounds (2), allows (2), without (2), assuming (2), parameters (2), auto (2), relative (2), get, started, design, website, name, email, write, loading, collapse, bar, manage, subscriptions, view, reader, content, privacy, already, account, now, blog, archive, effective, equilibrium, projections, resulting, ohu, range, layer, ecs, bias, local, feedbacks, patterns, gfdl, cm3, github, comparison, combining, instrumental, paleo, posts, zeke, hausfather, whiteboard, ron, broberg, steven, mosher, realclimate, open, mind, tamino, moyhu, nick, stokes, judith, curry, isaac, held, hyp, testing, andrew, ttca, charts, audit, chad, herman, blackboard, bart, verheggen, amac, air, vent, blogroll, regression, approach, detect, pause, stopped, factors, centurythe, greenhouse, effect, justify, your, pet, idea, millionnaire, tax, china, news, 2000m, ohc, window, alyce, how, statements, target, rcp4, rcp6, scenarios, evidence, christine, barr, briana, radiation, budget, pages, fun, residuals, mutli, exhibited, theoretically, respect, wouldn, physical, manifest, form, next, looks, similar, considerably, variation, either, almost, scattered, randomly, list, performing, comparable, comparisons, simulated, scenario, generating, various, consisting, sin, wave, 55th, shown, below, same, coloring, words, clear, thing, averaging, cancelling, settles, themselves, combination, match, when, field, turns, closer, fields, subject, ongoing, research, superficial, location, month, tend, symmetrically, single, consistently, however, mentioned, often, heard, outperformed, invidual, wasn, sure, necessarily, stumbled, across, where, digs, quote, terms, center, pack, impressive, low, even, determining, years, overall, later, instead, intended, purely, luck, none, good, job, guessing, apart, they, meant, small, intervals, supposed, swamp, changes, try, capture, correlations, something, diffs, total, set, technically, possibly, bars, dataset, happens, limiting, points, per, fit, suggest, gets, down, mostly, volcanic, eruptions, fact, surprising, speak, too, highly, histogram, ippc, above, root, square, stats, guy, off, wanted, couple, methods, determine, chose, metrics, baselining, obs, model_run, totally, guarantee, visa, versa, length, obviously, anything, suspect, leaves, gives, dive, into, work, programming, sres, a1b, downloaded, file, read, over, internet, planning, including, ability, additional, datasets, giss, format, bit, difficult, parse, need, borrow, kelly, scripts, help, explorer, been, noted, specifically, context, degree, reasons, twice, relevant, discussion, thinking, originally, path, you, top, tighter, stay, going, forward, estimate, predictions, treat, rogue, terrible, leading, ridiculously, large, generate, many, want, needing, simulation, negative, aspect, ar1, uncertain, simulating, trying, extract, show, wildly, yellow, bottom, creation, graph, seen, says, created, simulations, whereas, access, nonetheless, annual, anomalies, act, approximate, rnorm, standard, gleaned, compares, examining, functions, them, certainly, merely, higher, others, suprising, underlying, raise, hat, tip, arima, wondered, perhaps, able, captured, represents, instance, previous,

Text of the page (random words):
april 2011 troy s scratchpad troy s scratchpad april 29 2011 cmip3 model hindcasts and ar 1 weather noise filed under uncategorized troyca 6 14 pm data and code for this post here after looking at the hindcasts of the models in my previous post i wondered in that post if perhaps the differences between the mmm and each of the different model runs of the ensemble might be considered weather noise able to be captured as an ar 1 process assuming the mmm represents the actual forced component this would explain why it generally seems to perform better relative to actual observations which could be considered simply another instance of the forced component mmm ar 1 than any other individual runs well after examining the different model runs relative to the mmm and using the auto arima hat tip and or ar functions on them in r it certainly does not seem like these difference are merely the result of ar 1 noise some models generally seem to trend higher while others trend lower than the mmm which is not suprising given their underlying differences but it does raise the question again of why the mmm seems to perform better nonetheless the errors between the mmm and the hadcrut annual anomalies do seem to act like an ar 1 process using the approximate auto correlation of 0 5 and rnorm standard deviation of 0 1 that were gleaned from these errors the following image compares what it would look like with mmm weather noise of ar 1 vs just mmm and all the model runs in both graphs there were 54 different yellow runs the bottom image should basically be a re creation of the graph in the ipcc seen here in my last post although there are some differences the ipcc one says it was created from 58 simulations and 14 different models whereas what i had access to were 54 different ensemble members from 22 different models as you can see the top image shows tighter bounds but the observations still seem to stay within it for the 20th century to me going forward this would seem to be a better way to estimate noise in future predictions than simply showing all models as it allows us treat the mmm as the forced component without rogue terrible models leading to ridiculously large error bounds it also allows the chance to generate as many model runs as we want without needing a model simulation for each of course the negative aspect of assuming a noise model based on the errors between observations and the mmm is that those parameters for the ar1 process can be uncertain simulating a series and then trying to extract the parameters again can show some wildly different results lucia has relevant discussion on this here which got me thinking originally along this path comments 1 april 22 2011 cmip3 multi model mean vs individual runs in 20th century hindcast filed under uncategorized troyca 5 55 pm it has generally been noted that the multi model mean in ar4 outperforms the individual members of the ensemble in this post i ll specifically look at this within the context of the 20th century hindcast of surface temperatures to see to what degree this is the case as well as reasons why this might be the case after all the following figure appears twice in the ipcc ar4 report in chapter 8 and 9 like my last post this one gives me a chance to dive more into the cmip3 models and work on my time series programming in r once again i ll be looking at the 54 ensemble members for sres a1b that i got from climate explorer the code and data can all be downloaded here the r file itself will read the hadcrut data over the internet i was planning on including the ability to use additional surface temperature datasets for the 20th century at least giss but the format is a bit more difficult to parse i may need to borrow one of kelly o day s scripts to help with that processing in the future what is better not being a stats guy and i could be way off here i wanted to use a couple of simple methods to determine which hindcasts might be considered the best or closest to actual observations i chose two metrics 1 the rmse after baselining on the 1900 1950 average and 2 the correlation between the diff obs and diff model_run while 1 and 2 might not be totally independent getting a high score in 1 does not guarantee a high score in 2 and visa versa there s also a question of what length of time we should be looking at obviously anything less than 1 year averages is suspect but using a 30 year average only leaves us with 3 independent observations root mean square error the following histogram shows the rmse for 1 year average temperatures basically what we see in the ippc figure above by this metric the multi model mean does perform the best the linear trend value refers to the rmse of a linear trend line fit to the 20th century temperature and the flat line value refers to the rmse of a flat line at the 1900 1950 average value that the multi model mean does better than the simple linear trend seems to suggest that it gets some of the up and down fluctuations correct mostly around volcanic eruptions the fact that a simple flat line does not look worse than some of the runs is somewhat surprising and does not speak too highly of some of those runs what happens if we look at 10 year averages limiting us to only 10 points per run once again the multi model mean performs pretty well compared to the total set of runs and better than the linear trend but technically not the best although it is possibly within the errors bars based on uncertainty in the hadcrut dataset correlation of diffs what about if we try to capture the forced fluctuations minus the trend for 1 year the correlations look something this the multi model mean performs better than most of the individual runs although this may be purely by luck as none of the runs do a particularly good job of guessing year to year fluctuations apart from the trend of course they are not meant to weather noise at these small intervals are supposed to swamp any forced changes instead a better metric for the intended use might be the 10 year fluctuations in this case the multi model mean appears to be in the center of the pack what s more the r values are not particularly impressive given the low df even for determining the difference between 10 years minus the overall trend more on this later why does the multi model mean perform better at least in terms of rmse than individual models as i mentioned i d often heard it was the case that the mmm outperformed invidual models but i wasn t sure why this should necessarily be the case along the way i stumbled across this post from science of doom where he digs up the following quote from ipcc chapter 8 why the multi model mean field turns out to be closer to the observed than the fields in any of the individual models is the subject of ongoing research a superficial explanation is that at each location and for each month the model estimates tend to scatter around the correct value more or less symmetrically with no single model consistently closest to the observations this however does not explain why the results should scatter in this way in other words the explanation is not clear one thing we might consider is that there is a forced temperature response in the models but that each of the runs has different weather noise getting in the way and that the multi model mean with the averaging cancelling out most of the noise settles pretty well along around the forced response the observations themselves being simply another combination of this forced response weather noise will not have their noise match up well the noise from another run and so the rmse will be lower when compared to this multi model mean than to any individual run i simulated this scenario by generating various time series consisting of red noise sin wave linear trend the first 54 runs i consider the model the 55th i consider the observations and then the average of the first 54 is the multi model mean one example of the result is shown below using the same coloring as the ipcc run it looks somewhat similar to the actual hindcast with the multi model mean showing considerably less variation than either the observations or the individual model runs in almost all my runs the fake mmm had a lower rmse than any of the individual fake models compared against the fake observations but the r value of the diff d series was scattered randomly around the list not generally performing better or worse than any of the individual runs both of these properties seem comparable to the actual comparisons of models vs observations vs the multi model mean so if the residuals of the individual models minus the mutli model mean exhibited properties of an ar 1 process theoretically this would explain at least for surface temperatures why the mmm outperforms the individual runs with respect to rmse in the hindcast of course this still wouldn t be a physical explanation of why the deviation from the forced response would manifest itself in the form of red noise more on that in my next post just for fun based on the rmse for 1 year averages and the correlation of diff d values for 10 year averages here are the best two ensemble members and the multi model mean compared against the observed values comments 2 pages about radiation budget and climate sensitivity recent comments briana on comment on the phase relation between atmospheric carbon dioxide and global temperature christine barr on how well do the ipcc s statements about the 2 c target for rcp4 5 and rcp6 0 scenarios reflect the evidence alyce on comment on the phase relation between atmospheric carbon dioxide and global temperature window on estimating sensitivity from 0 2000m ohc and surface temperatures the case against a u s carbon tax us china news on on forcing enhancement efficacy and kummer and dessler 2014 millionnaire on cmip3 multi model mean vs individual runs in 20th century hindcast no the greenhouse effect does not justify your pet idea on on forcing enhancement efficacy and kummer and dessler 2014 warming has not stopped surface temperatures reflect multiple forcing factors the environmental centurythe environmental century on could the multiple regression approach detect a recent pause in global warming blogroll air vent amac bart verheggen blackboard lucia chad herman climate audit climate charts graphs k o day hyp testing andrew ttca isaac held judith curry moyhu nick stokes open mind tamino realclimate science of doom steven mosher whiteboard ron broberg zeke hausfather recent posts combining recent instrumental sensitivity estimates with paleo sensitivity estimates cmip5 processing on github with another comparison of observed temperatures to cmip5 model runs estimating ecs bias from local feedbacks and observed warming patterns example with gfdl cm3 on forcing enhancement efficacy and kummer and dessler 2014 effective vs equilibrium sensitivity uncertainty in projections resulting from the cmip5 ohu efficacy range in a two layer model search search for archive october 2014 september 2014 june 2014 may 2014 march 2014 february 2014 october 2013 september 2013 august 2013 july 2013 may 2013 april 2013 february 2013 january 2013 december 2012 november 2012 october 2012 september 2012 august 2012 july 2012 may 2012 april 2012 march 2012 february 2012 january 2012 december 2011 november 2011 october 2011 september 2011 august 2011 july 2011 june 2011 may 2011 april 2011 march 2011 february 2011 january 2011 december 2010 november 2010 august 2010 blog at wordpress com subscribe subscribed troy s scratchpad sign me up already have a wordpress com account log in now privacy troy s scratchpad subscribe subscribed sign up log in report this content view site in reader manage subscriptions collapse this bar loading comments write a comment email required name required website design a site like this with wordpress com get started