Meta tags:

Headings (most frequently used words):

data, applied, and, machine, learning, november, 2023, sites, to, blog, mining, science, friday, 24, thursday, 02, monday, may, 22, 2017, predictive, analytics, contributors, our, web, subscribe, this, smart, collective, archive, blogs, conferences, labels, insurance, popular, posts, how, confident, are, we, of, model, predictions, what, if, generative, ai, turns, out, be, dud,

Text of the page (most frequently used words):
the (98), and (56), data (52), mining (28), for (28), this (27), software (22), #analytics (21), that (21), have (21), from (20), are (19), com (16), you (16), with (15), model (15), predictive (15), what (15), programming (13), workflow (13), https (12), interface (12), one (11), but (11), may (11), november (11), tools (11), abbott (10), their (10), like (10), january (9), http (9), used (9), most (8), analysis (8), machine (8), learning (8), april (8), july (8), how (8), figure (8), these (8), they (8), functions (8), was (7), business (7), not (7), december (7), february (7), june (7), comments (7), dean (7), www (7), sas (7), use (7), models (7), will (6), posts (6), science (6), job (6), october (6), generative (6), all (6), png (6), some (6), visual (6), each (6), command (6), line (6), also (6), them (6), python (6), there (6), any (5), can (5), testing (5), first (5), between (5), survey (5), algorithms (5), august (5), march (5), style (5), has (5), user (5), way (5), tool (5), set (5), build (5), example (5), article (5), years (5), confidence (5), scores (5), training (4), why (4), over (4), posted (4), here (4), blog (4), statistics (4), prediction (4), modeling (4), matlab (4), september (4), out (4), applied (4), very (4), building (4), code (4), languages (4), commercial (4), because (4), would (4), into (4), still (4), same (4), your (4), several (4), using (4), provide (4), past (4), 100 (4), more (3), just (3), list (3), been (3), intelligence (3), popular (3), values (3), world (3), source (3), selection (3), missing (3), logistic (3), regression (3), metrics (3), understanding (3), conferences (3), books (3), top (3), sites (3), 2017 (3), 2023 (3), atom (3), flow (3), content (3), edge (3), alteryx (3), spss (3), advanced (3), blocks (3), learn (3), though (3), other (3), needed (3), difference (3), through (3), run (3), nodes (3), function (3), reasons (3), issue (3), day (3), kind (3), now (3), including (3), many (3), postings (3), few (3), published (3), r4stats (3), html (3), need (3), hope (3), post (3), marcus (3), agi (3), approach (3), really (3), bootstrap (3), intervals (3), part (2), problem (2), overfitting (2), without (2), than (2), poor (2), include (2), methods (2), performance (2), time (2), which (2), about (2), means (2), clementine (2), took (2), recently (2), course (2), insurance (2), trends (2), theory (2), statistical (2), rexer (2), random (2), principal (2), conference (2), open (2), integration (2), decision (2), preparation (2), contest (2), competition (2), critical (2), classification (2), kdnuggets (2), stats (2), research (2), dwinnell (2), 2013 (2), 2014 (2), 2015 (2), turns (2), dud (2), confident (2), subscribe (2), weka (2), net (2), images (2), cutting (2), uploads (2), valprop (2), rapidminer (2), miner (2), mpr (2), ibm (2), modeler (2), examples (2), workflows (2), packages (2), note (2), own (2), performing (2), much (2), existing (2), well (2), create (2), primarily (2), generation (2), clients (2), problems (2), sets (2), didn (2), write (2), larger (2), available (2), algorithm (2), new (2), important (2), reason (2), added (2), node (2), external (2), itself (2), replace (2), community (2), another (2), think (2), string (2), together (2), functional (2), made (2), language (2), want (2), file (2), mean (2), transform (2), steps (2), done (2), easily (2), did (2), manager (2), self (2), interfaces (2), see (2), exactly (2), options (2), won (2), typical (2), drop (2), down (2), myself (2), decades (2), early (2), those (2), pdf (2), trending (2), found (2), articles (2), google (2), identified (2), such (2), hurwitz (2), crm (2), xu13 (2), bip (2), patdws (2), updated (2), modelers (2), know (2), after (2), guessing (2), experienced (2), information (2), month (2), ceo (2), whom (2), linkedin (2), allgöwer (2), wikipedia (2), works (2), exist (2), however (2), never (2), topic (2), related (2), compute (2), interval (2), record (2), sample (2), baseline (2), score (2), when (2), sure (2), predictions (2), calculating (2), skip (2), explained, estimates, target, variable, regions, dangerous, accuracy, context, introductory, texts, substantial, coverage, various, assessing, true, perform, tracking, depending, whose, definition, reads, activities, comprise, vary, two, items, always, forgotten, step, children, question, clustering, thought, knew, answer, opportunity, prove, mys, normalization, matters, reminded, overlap, reader, similarities, differences, whimsical, webinars, uplift, trendiness, torture, theorist, theology, text, significance, special, speaking, spam, skills, sampling, roc, risk, regular, expressions, rare, events, deifition, privacy, components, component, practitioner, practice, powerpoint, plot, pca, parody, outliers, orgranizations, organizational, mse, morality, assessment, mac, compatibility, literature, link, kdd, kaggle, jobs, idempotnence, idempotent, humor, hate, graphs, graphing, graphics, graph, fundamentals, flash, versus, substance, feature, ethics, error, ensembles, eigenanalysis, economics, due, diligence, dorian, pyle, documentation, radio, diy, distributions, decisions, trees, visualization, reduction, users, perceptions, education, degree, evaluation, junctures, crisp, computer, competitions, chart, careers, career, objectives, bioinformatics, big, baseball, art, labels, meetings, alltop, news, feb09, airport, butler, ontime, flight, nuggets, kevin, hillstrom, minethatdata, tim, mann, blogs, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2016, pre, archive, smart, collective, our, web, contributors, home, older, knowledge, siliconafrica, themes, directorypress, thumbs, orange, blogger, googleusercontent, img, r29vz2xl, avvxsehgzlqeqxisyf_rg8s94ninncugcbcbbth, mdlglr, 7djw, e6tdci1awkxci7m2bjhtipvqmkyfqoqbakqckmahesr8xs9xamxiqvtiu0_vgr_, sumeps_q0d4upwyoy2kay6z3pw, s1600, orangepython, microsoft, azure, azurecomcdn, azureedge, cvt, f50521786d7494d1129069f701d8751293420aaad404b4aa8260a68606055807, page, services, simple, scalable, jpg, default, files, product, designer_3, processing, xomnia, rm_studio, 1024x551, enterprise, media, licdn, aaeaaqaaaaaaaakwaaaajgvjmge3yze4ltrhztetnge4zi05zwziltc3zdgwy2vjnzzizg, stream, 34f2c, cdn, softlayer, 8034f2c, dal05, auth_db1cfc7b, a055, 460b, 9274, 1fd3f11fe689, 5b0fd91ef0e1a6f21f6e983ccc775a37, offering_3d4b6acc, 2e09, 4451, b4d0, 3385f4385aa8, statistica, workshop, hands, acclaimed, shown, below, dominated, similar, builds, separates, novices, experts, awareness, particular, best, recommend, analysts, becoming, expert, its, fullest, potential, personally, regularly, fortran, scripting, unix, shell, csh, ksh, bash, mathematica, others, eventually, consulting, contained, solve, since, limited, sometimes, make, creative, accomplish, stop, being, successful, perspective, biggest, functionas, standpoint, manifested, obscure, typically, within, intent, isn, enhance, yet, especially, case, leading, support, already, ties, academic, whether, thing, sequence, commands, manipute, load, csv, positively, skewed, variables, log, split, subsets, then, tree, second, reuse, validation, exact, merely, connect, third, explain, visually, needing, understand, documenting, least, high, level, fair, considerable, customization, inside, evident, nevertheless, often, numerous, specify, call, successfully, even, excellent, menu, variety, legacy, appears, where, connects, icons, represent, tasks, introduce, package, khoros, cantata, 90s, paradigm, historically, windows, menus, embracing, built, ground, knime, embraced, beginnings, citeseerx, ist, psu, edu, viewdoc, download, doi, 9854, rep, rep1, type, interested, getting, provides, useful, rankings, number, led, java, consider, reveals, nearly, identical, increase, compared, decrease, overall, huge, install, base, recent, contains, additional, usually, comparisons, scholarly, scholar, hits, addition, summaries, surveys, polls, reviews, technology, companies, gartner, forrester, associates, thanks, providing, links, dam, en_us, doc, analystreport, 107212, global, sap, campaign, usa, index, urlid, pages, gartnermqadvancedanalyticsnowavailable, rexeranalytics, results, popularity, lists, doing, rather, gui, based, perhaps, importantly, free, robust, supported, broad, extensive, times, edited, predictiveanalyticsworld, patimes, 0408152, 5129, 2024, continues, value, intended, almost, posting, flagged, copyright, digital, rights, specific, wasn, left, screen, capture, products, safe, irony, vendor, websites, dont, current, certainly, smarterhq, monday, garymarcus, substack, turned, comes, across, consistent, closest, colleagues, who, seriously, doubt, serious, unsolved, core, ranging, tendency, confabulate, hallucinate, false, inability, reliably, wolfram, alpha, instability, makes, candidates, engineering, systems, everybody, industry, probably, believe, imminent, stokes, narrative, inevitability, drives, stock, prices, startup, valuations, dario, amodei, anthropic, projected, demis, hassabis, deepmind, projections, near, term, key, quotes, end, follow, twitter, different, sides, debates, yann, lecun, followed, gary, discovered, views, intriguing, thursday, compare, approaches, pros, cons, coming, weeks, stay, tuned, year, week, europe, talk, subject, given, michael, naatz, entitled, nformal, universal, method, uncertainty, quantification, introduced, accomplsihing, objective, summary, heard, experiment, limitations, might, org, wiki, conformal_prediction, allgoewer, machinelearningweek, fine, seen, formal, treatment, discredit, had, signfiicant, search, find, equivalent, identifying, probabilities, 5th, 95th, percentiles, ranks, interpolate, 2nd, 3rd, 97th, 98th, ranked, every, test, distribution, protocol, samples, gets, types, non, parametric, don, benefit, kinds, measures, decade, love, grounded, strong, assumptions, insight, coefficients, correct, linear, a7br3pa, stackexchange, questions, 354098, binary, neural, networks, xgboost, forests, etc, get, output, ranges, stable, does, friday, tips, tricks, topics, called, title, reflect, hosted, sidebar, main,

Text of the page (random words):
r each bootstrap sample using same the protocol as the baseline model run each model through the testing set we now have 100 scores for every record in the test set a distribution of scores compute the 90 confidence interval equivalent by identifying the probabilities or model scores at 5th and 95th percentiles ranks 5 and 95 of the 100 scores for the 95 confidence interval one would need to interpolate between 2nd and 3rd and also the 97th and 98th ranked scores this works fine for any algorithm however i d never seen a formal treatment of this topic this is really to my discredit as i had never really done a signfiicant search to find any theory related to this topic at this year s machine learning week europe https machinelearningweek eu there was a talk on this subject given by dr michael naatz allgöwer https www linkedin com in allgoewer entitled nformal prediction a universal method for uncertainty quantification that introduced another way of accomplsihing this objective a wikipedia summary of the approach is here https en wikipedia org wiki conformal_prediction i like what i ve heard from dr allgöwer at the conference and would like to experiment with this approach to learn how it works and what limitations might exist for the approach i hope to compare the approaches with pros and cons in the coming weeks stay tuned posted by dean abbott at 12 09 pm 0 comments thursday november 02 2023 what if generative ai turns out to be a dud i follow posts on twitter from different sides of the generative ai debates including yann lecun whom i ve followed for decades and gary marcus whom i discovered just in the past few years i ll post at some other time about my views but found this post by marcus to be intriguing i first published my comments here on linkedin key quotes at the end of the article everybody in industry would probably like you to believe that agi is imminent it stokes their narrative of inevitability and it drives their stock prices and startup valuations dario amodei ceo of anthropic recently projected that we will have agi in 2 3 years demis hassabis ceo of google deepmind has also made projections of near term agi i seriously doubt it we have not one but many serious unsolved problems at the core of generative ai ranging from their tendency to confabulate hallucinate false information to their inability to reliably interface with external tools like wolfram alpha to the instability from month to month which makes them poor candidates for engineering use in larger systems this is exactly how it comes across to me and is consistent with what i ve experienced myself and what my closest colleagues who have used generative ai have also experienced the marcus article https garymarcus substack com p what if generative ai turned out posted by dean abbott at 8 38 pm 0 comments monday may 22 2017 what programming do predictive modelers need to know dean abbott smarterhq and abbott analytics note this post almost 7 years after posting 5 22 2017 has been flagged for copyright or some kind of digital rights issue of course the specific issue wasn t identified so i m left guessing i m guessing it was a screen capture of one of the software products so to be safe i took them all out the irony is that the images are all from the vendor websites i dont think any of them are current i certainly hope not i hope the article continues to provide the value it was intended to provide dean 1 13 2024 first published in predictive analytics times http www predictiveanalyticsworld com patimes what programming do predictive modelers need to know 0408152 2 5129 2014 updated and edited here in most lists of the most popular software for doing data analysis statistics and predictive modeling the top software tools are python and r command line languages rather than gui based modeling packages there are several reasons for this perhaps most importantly that they are free they are robust programming languages supported by a very broad user community and they have extensive sets of algorithms one recent survey of software was published at r4stats com http r4stats com articles popularity and contains additional metrics not usually found in software comparisons including scholarly articles that include software google scholar hits and job trends in addition to the more typical summaries by user identified use of software such as with the rexer analytics surveys http www rexeranalytics com data miner survey results 2013 html and polls on kdnuggets com and reviews of software by technology research companies such as gartner http pages alteryx com gartnermqadvancedanalyticsnowavailable t html forrester http global sap com campaign na usa crm xu13 bip patdws index html urlid crm xu13 bip patdws and hurwitz associates http www sas com content dam sas en_us doc analystreport hurwitz advanced analytics 107212 pdf thanks to r4stats for providing these links in their article for those interested in getting a job in analytics the article provides very useful rankings of software by the number of job postings led by java sas python c c c r spss and matlab they also provide a few examples of the trending in the job postings of the tools over the past 7 years important trending to consider as well this reveals for example a nearly identical increase in python and r compared with a decrease in sas over the past few years sas is still 2 overall in job postings though because of the huge sas install base the user interface that appears to have won the day in commercial software for predictive modeling is the workflow style interface where a user connects icons that represent functions or tasks into a flow of functions this kind of interface has been in use for decades and one that i was first introduce to in the software package khoros cantata in the early 90s http citeseerx ist psu edu viewdoc download doi 10 1 1 22 9854 rep rep1 type pdf clementine was an early commercial tool using this paradigm now ibm spss modeler and now most tools including those that have historically used drop down windows style menus are embracing a workflow interface and it s not just commercial tools that are built from the ground up using a workflow style interface many open source software like knime and rapidminer embraced this style from their beginnings even though workflow interfaces have won the day there are several excellent software tools that still use a typical drop down file menu interface for a variety of reasons some legacy and some functional i still use several of them myself there are several reasons i like the workflow interface first it is self documenting much like command line interfaces you see exactly what you did in the analysis at least at a high level to be fair some of these nodes have considerable critical customization options set inside the nodes but the function is self evident nevertheless command line functions have the same issue there are often numerous options one has to specify to call a function successfully second you can reuse the workflow easily for example if you want to run your validation data through the exact same data preparation steps you used in building your models you merely connect a new data source to the workflow third you can explain what you did to your manager very easily and visually without the manager needing to understand code another way to think of the workflow interface is as a visual programming interface you string together functional blocks from a list of functions nodes made available to you by the software so whether you build an analysis in a visual workflow or a command line programming language you still do the same thing string together a sequence of commands to manipute and model the data for example you may want to load a csv file replace missing values with the mean transform your positively skewed variables with a log transform split you data into training and testing subsets then build a decision tree each of these steps can be done with a node visual programming or a function programming from this perspective the biggest difference between visual programming and command line programming is that r and python have a larger set of functionas available to you from an algorithm standpoint this difference is primarily manifested in obscure or new cutting edge algorithms this is one important reason why most visual programming interface tools have added r and python integration into their software typically through a node that will run the external code within the workflow itself the intent isn t to replace the software but to enhance it with functions not yet added to the software itself this is especially the case with leading edge algorithms that have support in r or python already because of their ties with the academic community personally i used to create code regularly for building models primarily in c and fortran 3 rd generation languages though also in other scripting 4 th generation languages like unix shell programming sh csh ksh bash matlab mathematica and others but eventually i used commercial software tools because my consulting clients used them and they contained most of what i needed to do to solve data mining problems since they all have limited sets of functions i would have to sometimes make creative use of the existing functions to accomplish what i needed to do but it didn t stop me from being successful with them and i didn t have to write and re write code for each of these clients each of these tools have their own way of performing an analysis command line tools also have their own way of performing an analysis much of what separates novices and experts in a software tool is not an awareness of the particular functions or building blocks but an understanding of how best to use the existing building blocks this is why i recommend analysts learn a tool and learn it well becoming an expert in the tool so that the tool is used to its fullest potential examples of workflows in some of the most popular and acclaimed advanced analytics software packages are shown below note that the style that has dominated these top tools is the visual programming interface and they are very similar in how the user builds these workflows figure 2 statistica workflow from my predictive analytics world workshop advanced methods hands on figure 3 ibm spss modeler stream from https 34f2c https cdn softlayer net 8034f2c dal05 v1 auth_db1cfc7b a055 460b 9274 1fd3f11fe689 5b0fd91ef0e1a6f21f6e983ccc775a37 offering_3d4b6acc 2e09 4451 b4d0 3385f4385aa8 png figure 4 sas enterprise miner from https media licdn com mpr mpr aaeaaqaaaaaaaakwaaaajgvjmge3yze4ltrhztetnge4zi05zwziltc3zdgwy2vjnzzizg png figure 5 rapidminer processing flow from http www xomnia com wp content uploads 2015 03 rm_studio 1024x551 png figure 6 alteryx workflow from https www alteryx com sites default files uploads valprop valprop product designer_3 png figure 7 microsoft azure workflow from https azurecomcdn azureedge net cvt f50521786d7494d1129069f701d8751293420aaad404b4aa8260a68606055807 images page services machine learning simple scalable cutting edge jpg figure 8 orange data mining from https blogger googleusercontent com img b r29vz2xl avvxsehgzlqeqxisyf_rg8s94ninncugcbcbbth mdlglr 7djw e6tdci1awkxci7m2bjhtipvqmkyfqoqbakqckmahesr8xs9xamxiqvtiu0_vgr_ sumeps_q0d4upwyoy2kay6z3pw s1600 orangepython png figure 9 weka knowledge flow from http www siliconafrica com wp content themes directorypress thumbs weka png posted by dean abbott at 8 00 pm 0 comments older posts home subscribe to posts atom applied predictive analytics contributors dean abbott will dwinnell our web sites abbott analytics will s data mining in matlab subscribe to this blog posts atom posts all comments atom all comments smart data collective blog archive 2023 2 november 2 how confident are we of machine learning model pre what if generative ai turns out to be a dud 2017 2 may 1 march 1 2016 1 april 1 2015 2 december 1 july 1 2014 5 july 1 may 2 january 2 2013 13 november 1 september 1 august 1 july 1 june 2 april 2 february 3 january 2 2012 17 december 1 november 1 october 1 september 2 august 2 july 1 june 1 may 1 april 4 february 2 january 1 2011 17 december 1 november 1 july 1 june 1 may 1 april 3 march 2 february 4 january 3 2010 32 november 1 october 6 september 3 august 4 july 1 june 3 may 6 february 3 january 5 2009 23 december 4 november 2 october 1 july 1 june 1 may 1 april 3 march 5 february 3 january 2 2008 14 november 1 october 2 september 1 may 2 april 7 january 1 2007 42 december 1 november 1 october 3 august 5 july 4 june 2 may 4 april 4 march 4 february 5 january 9 2006 18 december 4 november 8 october 6 2005 6 june 1 may 1 april 1 march 1 february 1 january 1 2004 1 december 1 2003 1 august 1 data mining blogs and sites data mining in matlab will dwinnell tim mann s data mining blog data mining research kevin hillstrom s minethatdata kd nuggets abbott analytics airport butler ontime flight stats predictive analytics world feb09 alltop top data mining news data mining conferences kdnuggets list of meetings labels algorithms 2 analytics 1 art 1 baseball 3 big data 2 bioinformatics 1 books 3 business 1 business analytics 2 business intelligence 4 business objectives 1 business understanding 1 career 1 careers 1 chart 1 classification 2 competition 1 competitions 1 computer science 1 conferences 4 contest 1 crisp dm 1 critical junctures 1 data evaluation 1 data mining 18 data mining books 4 data mining competition 1 data mining conferences 4 data mining contest 1 data mining data 1 data mining degree 1 data mining education 1 data mining perceptions 1 data mining software 2 data mining survey 1 data mining training 1 data mining users 1 data mining vs statistics 1 data preparation 4 data reduction 1 data science 1 data selection 1 data understanding 2 data visualization 2 decision trees 1 decisions 1 distributions 1 diy 1 dm radio 1 do 1 do not 1 documentation 1 dorian pyle 1 due diligence 1 economics 1 eigenanalysis 1 ensembles 1 error metrics 1 ethics 1 feature selection 1 flash versus substance 1 fundamentals 1 graph 1 graphics 3 graphing 1 graphs 1 hate 1 humor 2 idempotent 1 idempotnence 1 insurance 1 integration 1 job 1 jobs 2 kaggle 1 kdd 2 link analysis 2 literature 1 logistic regression 1 mac pc compatibility 1 machine learning 1 missing data 2 missing values 2 model assessment 1 model performance 1 model selection 3 modeling 3 morality 1 mse 1 open source 1 organizational 1 orgranizations 1 outliers 1 parody 1 pca 1 plot 1 powerpoint 1 practice 1 practitioner 1 prediction 1 predictive 1 predictive analytics 2 predictive analytics conference 1 predictive analytics world 1 principal component 1 principal components analysis 1 privacy 2 problem deifition 1 programming 2 r 1 random 1 rare events 1 regular expressions 1 rexer analytics 1 risk 1 roc 1 sampling 3 science 1 skills 1 software 5 spam 1 speaking 1 special values 1 statistical signific...