Meta tags:

Headings (most frequently used words):

json, and, safe, format, versioning, automatic, migration, using, setversion, removeversion, fromjson, tojson, instances, examples, acknowledgments, modules, downloads, readme, for, why, how, does, it, work, keep, in, mind, fresh, start, production, some, effects, of, changing, message, safejson, migrate, testing, noversion, non, object, no, down, time, cheatsheet, maintainer, corner, version, kind, type, name, safefrom, safeto, reverse, backstory,

Text of the page (most frequently used words):
the (219), and (90), type (75), this (56), json (53), #safejson (51), #version (50), migrate (47), data (41), instance (38), from (33), where (31), use (30), mytype (29), can (29), tojson (28), you (27), new (26), that (26), message (26), using (24), will (24), for (24), thirdtype (22), age (21), format (21), your (20), instances (20), migration (19), are (18), fromjson (18), services (17), with (17), reverse (17), name (17), safe (17), when (17), not (17), message_v0 (17), have (16), kind (16), secondtype (16), one (15), value (15), used (14), which (14), object (14), versioning (14), only (13), functions (13), has (13), firsttype (13), how (13), safefrom (13), chain (13), all (12), parsing (12), definition (12), parsejson (12), safefromjson (12), migratefrom (12), text (11), newtype (11), branch (10), aeson (10), person (10), personalinfo (10), newer (10), might (10), oldtype (10), safeto (10), package (10), library (9), formats (9), doe (9), would (9), parse (9), setversion (9), make (8), two (8), added (8), like (8), following (8), just (8), types (8), change (8), extension (8), noversion (8), address (8), phonenumber (8), but (8), firstname (8), lastname (8), example (8), some (8), string (8), ttfirstname (8), ttlastname (8), ttage (8), defined (8), fields (8), there (8), defining (8), code (7), still (7), production (7), messages (7), since (7), tag (7), older (7), any (7), different (7), versions (7), removeversion (7), keep (6), after (6), running (6), should (6), point (6), something (6), define (6), add (6), other (6), already (6), command (6), mage (6), withobject (6), return (6), while (6), then (6), current (6), want (6), deriving (6), show (6), maybe (6), 0000 (6), number (6), need (6), way (6), typ (6), also (6), field (6), these (6), set (6), possible (6), least (6), migrated (6), ghc (6), github (5), update (5), now (5), work (5), before (5), methods (5), down (5), time (5), our (5), safetojson (5), old (5), int (5), null (5), johnny (5), advised (5), does (5), don (5), parsed (5), defines (5), readme (5), com (5), cabal (4), hackage (4), ones (4), between (4), rename (4), mytype_old (4), extended_base (4), create (4), language (4), mid (4), mcommand (4), mperson (4), maddress (4), mphonenumber (4), msgid (4), msgcommand (4), msgdata (4), piperson (4), piage (4), piaddress (4), piphonenumber (4), back (4), migrating (4), instead (4), sure (4), start (4), able (4), bottom (4), been (4), txt (4), result (4), resulting (4), case (4), program (4), base (4), why (4), such (4), cases (4), non (4), more (4), remove (4), xxx (4), given (4), being (4), etc (4), testing (4), those (4), likely (4), https (4), vlix (4), they (3), original (3), them (3), ready (3), everything (3), both (3), recordwildcards (3), without (3), encode (3), extended_ (3), makes (3), going (3), through (3), moment (3), handle (3), uuid (3), versioned (3), included (3), adding (3), easy (3), guard (3), response (3), import (3), right (3), thirdtypehandler (3), either (3), database (3), include (3), fresh (3), setting (3), examples (3), overhead (3), than (3), incomingjson (3), get (3), vnil (3), completely (3), values (3), here (3), backward (3), compatibility (3), most (3), last (3), classes (3), automatic (3), description (3), 2026 (3), nideco (3), felix (3), paulusma (3), tasty (3), stackage (2), acknowledgments (2), receive (2), servers (2), its (2), updated (2), business (2), logic (2), development (2), changes (2), what (2), needed (2), cheatsheet (2), structure (2), having (2), expecting (2), out (2), errors (2), received (2), probably (2), master (2), represent (2), first (2), 00000000 (2), 000000000000 (2), add_user (2), john (2), middlename (2), street (2), steenstraat (2), addition (2), city (2), koekel (2), country (2), friesland (2), started (2), arbitrary (2), things (2), seem (2), contrived (2), val (2), jonathan (2), shelley (2), doegan (2), anita (2), mcdoe (2), http (2), look (2), res (2), eitherdecode (2), err (2), tts (2), body (2), request (2), function (2), module (2), bytestring (2), parsevalues (2), depending (2), immediately (2), qualified (2), rest (2), isspace (2), was (2), give (2), into (2), min (2), bytes (2), recursively (2), vals (2), my_type_field1 (2), my_type_field2 (2), guaranteed (2), requirement (2), preferable (2), doesn (2), incoming (2), internally (2), because (2), uses (2), correct (2), always (2), appropriate (2), long (2), though (2), soon (2), even (2), forward (2), omit (2), integer (2), method (2), whenever (2), parsable (2), didn (2), forget (2), test (2), note (2), mind (2), support (2), further (2), directly (2), definitions (2), extended_extension (2), actually (2), read (2), class (2), default (2), making (2), made (2), depends (2), unique (2), identifier (2), migratable (2), mainly (2), consists (2), endpoints (2), needing (2), effects (2), changing (2), aims (2), updating (2), contents (2), keeping (2), painless (2), achieved (2), log (2), rating (2), downloads (2), source (2), revised (2), maintainer (2), mit (2), containers (2), maintainers (2), revisions (2), browse (2), produced, core, inspired, david, himmelstrup, felipe, lessa, found, safecopy, enjoy, currently, branches, mytype_new, mytype_v0, copy, their, overwrite, copied, eachother, representation, separate, switch, preferably, everywhere, codebase, unless, explicitly, purposes, optional, previous, usages, worry, about, rerouting, message_old, implemented, roll, fear, anything, throwing, replace, anywhere, sent, decode, assuming, modified, temporary, let, call, important, system, deployed, functioning, ensures, floating, around, corresponding, messaging, live, hit, haskell, typeapplications, foo, httplbs, therequest, transformation, responsebody, left, putstrln, bad, requested, mapm, parseeither, saved, three, far, problem, char, frommaybe, break, dropwhile, nothing, sets, went, well, couple, weeks, wanted, whatever, noticed, had, people, backstory, working, once, second, starting, general, low, entire, individual, really, necessary, described, relative, respectively, hand, produce, removed, conversely, ideally, leaving, application, platform, visible, outside, world, parsemaybe, wrong, array_of_unversioned_mytypes, adds, difference, removes, recommended, implementation, warned, top, level, certain, overall, course, present, sometimes, desireable, third, parties, customers, impose, onto, insert, override, caution, own, risk, best, constraints, dropped, thus, versionless, attempted, match, alone, try, lead, succesful, reason, versionlesstype, switching, literal, primitive, possibility, less, advise, migrations, adjust, migratereverseroundtripprop, migrateroundtripprop, contains, very, useful, ensure, consistent, inconsistencies, failed, testconsistency, points, take, extended, fails, attempts, above, option, higher, done, kinds, encountered, magic, happens, converted, few, convenience, lot, easier, experience, similar, containwithobject, contain, valid, desirable, sub, inside, happen, interface, variants, times, reporting, pre, easily, typename0, objectprofile, means, next, future, tried, convenient, programs, phased, coming, trying, numbers, subsequently, none, regularly, seen, four, integrating, therefore, chosen, conventional, clash, existing, recover, potential, clashes, avoided, required, accomplished, adjusting, wrapped, custom, somevalues, true, extra, called, implication, additional, literals, share, same, assumes, basic, typeable, implement, implementations, exceptions, identify, typename, leave, goal, downtime, undesirable, often, place, cause, vice, versa, countered, creating, routed, correctly, includes, semantics, logged, normal, operation, results, excluded, queries, over, manually, come, question, restructured, otherwise, changed, mild, serious, headaches, architecture, communicating, obvious, environment, communication, micro, docs, available, success, reported, reports, build, status, votes, estimated, bayesian, average, 3298, total, days, ltshaskell, distributions, head, git, clone, repo, issues, bug, tracker, home, page, category, 25t21, 43z, revision, 04t12, 59z, uploaded, gmail, author, 2019, copyright, license, tested, details, vector, unordered, quickcheck, hunit, scientific, hashable, dlist, dependencies, changelog, info, rss, candidates, edit, information, trustees, corner, tarball, unpack, including, metadata, tar, quick, jump, index, modules, skip, depth, explanation, please, see, serialized, stay, report, vulnerability, propose, tags, user, accounts, upload, search,

Text of the page (random words):
0 a few convenience functions have been added to make defining safefrom and safeto methods a lot easier and to make the experience more similar to defining parsejson and tojson e g containwithobject etc migrate the migrate class is where the magic happens here you define which type can be converted into which other type when defining a migration from an older type to a newer type it s as easy as defining data oldtype oldtype text data newtype newtype text instance migrate newtype where read this as type migratefrom a b the type you migrate from to the a is b or b is the type you migrate from to get a type migratefrom newtype oldtype here we define how to change the structure from one type to the other migrate oldtype txt newtype txt now whenever json is encountered that should be parsed as an oldtype we can parse it as such and then immediately migrate it to newtype which is the one the program actually uses do not forget to set the kind of newtype to either extension or extended_extension to make use of this type of migration reverse migration there is also the option to support a migration from one version higher up in the chain to the current version this is done by defining the kind of the type in the safejson instance as one of the extended_ kinds and defining a migrate reverse a instance for the current type in that case a definition might look something like this using the above data definitions of oldtype and newtype instance migrate reverse oldtype where type migratefrom reverse oldtype newtype migrate newtype reverse oldtype migrate newtype t _ reverse oldtype t n b at the moment there is no support for extended migrating from newer versions further than the one directly defined in the type s reverse migrate instance i e if the parsing of the type defined in type a s migratefrom reverse a fails the other attempts will go down the chain not further up keep in mind here are some points to take note of when using this library testing the module data safejson test contains very useful testing functions to ensure your types and instances are consistent it is advised to at least testconsistency of any type you create a safejson instance for this makes sure you don t have any inconsistencies in your migration chain which would result in failed parsing of your type s i would also advise to use migrateroundtripprop or migratereverseroundtripprop whenever you add new migrations to be sure they are parsable and that you didn t forget to adjust the kind definition in the safejson instances using noversion there is a way to omit the version tag it s by using noversion instead of an integer literal when defining the version method this is used to not add a version tag to primitive values int text etc and also to give the possibility of adding a version less format to a migration chain when switching to safejson you might already have a json format in use adding this type as the bottom of a chain is as easy as instance safejson versionlesstype where version noversion kind extended_base or base if you don t need the forward migration if you include a noversion vnil in your chain it is advised to remove the need to include it as soon as possible since if the json being parsed has no version because it might be a completely different message having a noversion type in your chain alone will make it try to parse it as such in some cases this might lead to a succesful parse even though it s a completely different json message for that reason it is advised to remove the vnil from your chain as soon as possible as long as there is a version number in the json though vnil will not be attempted to be parsed since a version doesn t match no version using setversion and removeversion these functions are new in safe json 1 0 0 since the fromjson and tojson constraints have been dropped from safejson s definition and thus it is not guaranteed you can use data aeson functions to handle versionless json value s caution use these functions at your own risk it is always best to use versioning if possible setversion will insert override the given type s version in the given json value removeversion will remove all the safejson versioning from the json value setversion of course you d always like to have the correct version present in your value s but sometimes this is not appropriate or desireable one example would be when parsing incoming json from third parties e g customers which you don t want to impose the safejson versioning onto be warned this does only set the version at the top level setversion does not recursively set versions i e if your safefrom definition for example uses safefromjson when parsing certain fields these fields will not get the correct version from using setversion on the overall json value in these cases it is recommended to use a fromjson instance which doesn t use safefromjson in its definition for your incoming type instead of safejson you can still use safejson internally because if the type has a fromjson instance the safejson instance can just use that implementation for the safefrom definition this way you are guaranteed the only difference between safefromjson and parsejson is the requirement of a version field in safefromjson s case and it removes the need to use setversion which is preferable given data mytype instance safejson mytype where version 0 tojson adds no version fields incomingjson tojson mytype mytype wrong this will not parse using safejson functions λ encode setversion mytype incomingjson v 0 d array_of_unversioned_mytypes right this will parse using safejson functions λ just vals parsemaybe safefromjson parsejson incomingjson maybe value λ encode setversion mytype vals my_type_field1 xxx my_type_field2 xxx v 0 my_type_field1 xxx my_type_field2 xxx v 0 removeversion conversely removeversion should ideally only be used when the json is leaving your application or platform and you don t want the safejson versioning to be visible to the outside world removeversion on the other hand does remove all version fields recursively so removeversion safetojson will produce a json value with all v v and d fields removed non object versioning as described in version the version tag added to a non object value has more relative overhead than the tag added to a json object min 14 bytes and min 7 bytes respectively to keep general overhead low it is advised to version your entire message and only version individual fields if really necessary examples i want to give two example use cases for safejson the first is a fresh start with safejson and how you can migrate different versions at once the second is starting from a json format without versioning and then migrating into safejson in a production setting fresh start this is an arbitrary example some things might seem contrived the data types we re working with data firsttype firsttype text deriving eq show data secondtype secondtype text maybe int deriving eq show data thirdtype thirdtype ttfirstname text ttlastname text ttage int deriving eq show fromjson and tojson instances are included at the bottom of this example backstory we ve started using our program with firsttype and that went well for a couple of weeks then we wanted to add a field to maybe include the age of whatever the data was if it was something that has an age this is the secondtype a while after we noticed we only had data on people so why not represent it as such thirdtype json formats with versions added firsttype type mytype data johnny doe v 0 secondtype type mytype name johnny doe age 27 v 1 thirdtype type mytype firstname johnny lastname doe age 27 v 2 safejson instances this sets version to 0 and kind to base instance safejson firsttype instance safejson secondtype where version 1 kind extension instance migrate secondtype where type migratefrom secondtype firsttype migrate firsttype name secondtype name nothing import qualified data char as c import qualified data text as t instance safejson thirdtype where version 2 kind extension instance migrate thirdtype where type migratefrom thirdtype secondtype migrate secondtype name mage thirdtype ttfirstname firstname ttlastname lastname ttage frommaybe 1 mage where firstname rest t break c isspace name lastname t dropwhile c isspace rest our database in which we saved our data in json now has three different json formats depending on how far back in time we go but that s no problem for us we just request all the json from the database which result in a value and if we want to use it immediately just use something like the following parsevalues value either string thirdtype parsevalues mapm parseeither safefromjson but we can also use the value in the response body of a http request when requested by a different program and that program can just use a function from the data aeson safe module like eitherdecode to parse the bytestring body language typeapplications import data aeson safe as safe foo do res httplbs therequest since we ve defined the instance for safejson and migrate firsttype and secondtype can now migrate to thirdtype resulting in right thirdtype in this transformation case safe eitherdecode thirdtype responsebody res of left err putstrln bad value in response err right tts thirdtypehandler tts thirdtypehandler thirdtype io thirdtypehandler the http response would maybe look something like this type mytype data johnny doe v 0 type mytype name jonathan doe age null v 1 type mytype name shelley doegan age 27 v 1 type mytype firstname anita lastname mcdoe age 26 v 2 which would result in the following haskell data thirdtype ttfirstname johnny ttlastname doe ttage 1 thirdtype ttfirstname jonathan ttlastname doe ttage 1 thirdtype ttfirstname shelley ttlastname doegan ttage 27 thirdtype ttfirstname anita ttlastname mcdoe ttage 26 fromjson and tojson instances instance tojson firsttype where tojson firsttype txt object type string mytype data txt instance fromjson firsttype where parsejson withobject firsttype o do typ o type guard typ string mytype val o data return firsttype val instance tojson secondtype where tojson secondtype name age object type string mytype name name age age instance fromjson secondtype where parsejson withobject secondtype o do typ o type guard typ string mytype name o name age o age return secondtype name age language recordwildcards instance tojson thirdtype where tojson thirdtype object type string mytype firstname ttfirstname lastname ttlastname age ttage instance fromjson thirdtype where parsejson withobject thirdtype o do typ o type guard typ string mytype ttfirstname o firstname ttlastname o lastname ttage o age return thirdtype production migration this is an arbitrary example some things might seem contrived we ve started using json without versioning as a messaging format between live services adding to it has been easy but we ve hit a point where we need to change the format in a way that current services would not be able to parse them the data type already in production id 00000000 0000 0000 0000 000000000000 command add_user person firstname john middlename null lastname doe age 45 address street steenstraat number 25 addition a city koekel country friesland phonenumber null data message message mid uuid mcommand text mperson person mage int maddress address mphonenumber maybe phonenumber deriving eq show fromjson and tojson instances are included at the bottom of this example the format we want to change to v 0 id 00000000 0000 0000 0000 000000000000 command add_user data person firstname john middlename null lastname doe age 45 address street steenstraat number 25 addition a city koekel country friesland phonenumber null we ll represent this as the first versioned type of this message data message_v0 message_v0 msgid uuid msgcommand text msgdata personalinfo deriving eq show data personalinfo personalinfo piperson person piage int piaddress address piphonenumber maybe phonenumber deriving eq show we ll then create the safejson and corresponding migrate instances language recordwildcards instance safejson message where this is important since our old type has no version tag version noversion extended_ makes sure we can migrate from the newer version back to this one since the newer formats will start going through the system the moment the new services are deployed and we want the older services to keep functioning kind extended_base instance migrate reverse message where type migratefrom reverse message message_v0 migrate message_v0 reverse message msgid msgcommand piperson piage piaddress piphonenumber where personalinfo msgdata instance safejson message_v0 where version 0 extension ensures the new services will be able to handle any old formats still floating around kind extension instance migrate message_v0 where type migratefrom message_v0 message migrate message message_v0 mid mcommand person where person personalinfo mperson mage maddress mphonenumber assuming we ve added message_v0 on a new development branch and modified the business logic to use message_v0 at this point we create a temporary branch from the production branch probably master and let s call it something like master message migration on this branch we only add message_v0 with the safejson and migrate instances and replace the json functions with safejson ones for anywhere message is received by or sent to the current services i e use decode encode from data aeson safe instead of data aeson or safefromjson safetojson instead of parsejson tojson then we update the current services so they re ready to use safejson for migrating the new json formats we re expecting after the services that have safejson implemented are the only ones running we can roll out the new format without fear of anything throwing parsing errors after that we rename message to message_old and all usages of message_v0 to message and we re back to using message in our code but now with a new structure all while not having to worry about rerouting messages or down time fromjson and tojson instances language recordwildcards instance tojson message where tojson message object id mid command mcommand person mperson age mage address maddress phonenumber mphonenumber instance fromjson message where parsejson withobject message o do mid o id mcommand o command mperson o person mage o age maddress o address mphonenumber o phonenumber return message instance tojson message_v0 where tojson message_v0 object id msgid command msgcommand data msgdata instance fromjson message_v0 where parsejson withobject message_v0 o msgid o id msgcommand o command msgdata o data return message_v0 instance tojson personalinfo where tojson personalinfo object person piperson age piage address piaddress phonenumber piphonenumber instance fromjson personalinfo where parsejson withobject personalinfo o piperson o person piage o age piaddress o address piphonenumber o phonenumber return personalinfo no down time cheatsheet from not using safejson to using safejson add safe...