Meta tags:
description=C1 to C9;

Headings (most frequently used words):

spotmarket, k162space, graph, sample, empires, of, eve, dev, c1, to, c9, recent, posts, categories, archives, blogroll, tools,

Text of the page (most frequently used words):
the (105), and (41), for (28), that (22), have (20), data (16), #spotmarket (15), this (14), 2016 (14), eve (12), 2010 (12), 2011 (12), 2012 (12), 2014 (11), 2013 (10), can (10), with (10), enhancement (10), typeid (10), python (9), insert (9), table (8), was (8), domain (8), k162space (7), october (7), december (7), january (7), market (7), consumer (7), what (7), api (7), out (7), some (7), items (7), august (6), february (6), march (6), april (6), june (6), release (6), any (6), zkillboard (6), crest (6), use (6), project (6), lot (6), been (6), list (6), add (6), moon (6), comment (5), create (5), database (5), 2009 (5), november (5), may (5), july (5), more (5), about (5), flask (5), frontend (5), json (5), postgresql (5), will (5), filed (5), under (5), blake (5), author (5), posted (5), each (5), not (5), started (5), into (5), write (4), report (4), design (4), september (4), learning (4), how (4), proof (4), killmails (4), graphs (4), rather (4), different (4), right (4), from (4), get (4), query (4), kills (4), rec (4), sec (4), after (4), process (4), backend (4), check (4), you (4), also (4), here (4), mineral (4), change (4), timestamp (4), new (4), work (4), website (3), wordpress (3), com (3), now (3), blog (3), 2015 (3), nullsec (3), dev (3), empires (3), graph (3), found (3), pandas (3), developing (3), only (3), prices (3), than (3), currently (3), platform (3), next (3), there (3), but (3), put (3), using (3), item (3), over (3), key (3), when (3), replaced (3), service (3), has (3), end (3), want (3), keep (3), side (3), tags (3), web (3), import (3), etc (3), jsonb (3), same (3), dataset (3), datatype (3), post (3), killmail (3), where (3), were (3), sanity (3), page (3), speculation (3), first (3), pass (3), order (3), second (3), multiple (3), jumps (3), required (2), comments (2), log (2), sign (2), subscribed (2), subscribe (2), join (2), other (2), free (2), wormhole (2), pvp (2), pos (2), history (2), eveonline (2), sample (2), entries (2), branch (2), docker (2), good (2), time (2), indexes (2), finding (2), trends (2), concept (2), region (2), learn (2), build (2), parameterize (2), charts (2), migrating (2), graphing (2), bootstrap (2), style (2), jinja2 (2), really (2), bring (2), guide (2), them (2), basics (2), all (2), state (2), true (2), presentation (2), needs (2), primary (2), constraints (2), error (2), checking (2), like (2), speed (2), update (2), version (2), improved (2), got (2), kept (2), original (2), component (2), moved (2), could (2), impact (2), front (2), trading (2), everything (2), last (2), finished (2), down (2), coding (2), control (2), minutes (2), testing (2), install (2), enough (2), made (2), two (2), support (2), accurate (2), feature (2), step (2), used (2), wanted (2), expansion (2), ccp (2), which (2), object (2), verified (2), showing (2), built (2), previous (2), utc (2), mental (2), make (2), index (2), solarsystemid (2), added (2), growing (2), overview (2), leave (2), rens (2), jita (2), dodixie (2), amarr (2), are (2), finish (2), endpoint (2), adding (2), isn (2), records (2), wallet (2), scan (2), 300 (2), jobs (2), times (2), concurrent (2), futures (2), worker (2), month (2), regions (2), fix (2), duplicate (2), via (2), reworking (2), off (2), cacheduntil (2), small (2), ratting (2), kill (2), rates (2), need (2), npc (2), rows (2), name, email, loading, collapse, bar, manage, subscriptions, view, site, reader, content, privacy, already, account, subscribers, mid, dotlan, maps, chruker, tools, altruist, serpentine, marketsforisk, lovely, little, vagabond, jester, trek, inner, sanctum, ninveah, finders, keepers, evehermit, resurrection, prosper, opportunist, online, wormholes, manufacturing, fail, ardent, defender, blogroll, archives, uncategorized, ships, screenshot, industry, featured, failmail, dust514, aideron, robotics, categories, recent, posts, tweets, older, latest, updates, image, reach, polish, publish, leveraging, power, spending, thinking, problems, covariance, develop, takes, regionid, input, reporting, solar, system, height, length, percentages, finite, pixel, amounts, supports, many, formats, look, horrible, tablet, mobile, device, deciding, partner, help, polishing, needed, bricks, call, place, return, manipulated, modern, punch, current, pgadmin, faster, generation, app, run, host, port, debug, threaded, multithreading, receives, restful, before, 3128, 2948, rewrote, statements, perform, selecting, row, exists, then, inserting, does, statement, would, utilize, gain, even, further, come, upsert, inserts, laundry, improvements, highlights, luckily, principal, keeping, every, communicate, isolated, transparent, layer, neighboring, services, limitations, james, who, wrote, program, pushing, towards, angularjs, framework, goal, container, anyone, anywhere, deploy, aim, push, button, receive, deployment, let, wrangling, minimum, four, weeks, busy, development, cycle, working, become, conformable, restructured, vision, final, architecture, rapid, affirmative, sansha, r64, full, slated, main, focus, going, putting, letting, enable, disable, watch, todo, bad, correcting, instructions, 2982, 2850, 124, choice, later, night, while, logging, noticed, improvement, numbers, comparison, being, virtual, machines, had, access, amount, cpu, ram, results, writing, parser, decided, store, itself, simplicity, proved, path, did, prefered, lacked, fancy, inherit, above, following, convinced, migrate, upgrade, gathered, examples, anyalisis, long, flight, code, together, coordinates, parse, implementation, proven, included, manual, separate, isolate, datasets, rubicon, introduced, siphon, units, anchored, near, slowly, leech, within, shields, extractor, destroyed, show, combined, location, exposed, parallax, locate, nearest, celestial, give, contained, secret, affinity, gone, far, chart, regional, density, skews, type, placed, class, r16, minerals, bug, markethistory, 404, postgres, functions, date, format, iso, standardize, formatting, supervisor, persistent, paging, resume, recorded, basic, exception, handling, creation, ship, whew, big, direction, usability, cleanup, existing, moongoo, beginnings, flowing, cleaned, directions, try, easier, plex, fountain, copy, arrived, today, unboxing, shots, hardcover, edition, purchase, quality, artwork, stands, don, jquery, knowledge, buttons, yet, ability, back, interaction, high, foot, track, investments, getting, elegant, throw, transactionid, transactions, balances, replicate, features, start, integrate, character, information, processes, orders, blueprints, journal, assets, characters, portion, throwing, entire, response, relying, dedupe, scaled, workers, cores, able, interesting, submitted, returned, because, they, spodumain, pyerite, tritanium, hedbergite, arkonor, plagioclase, kernite, mexallon, requests, scanner, lets, asynchronously, execute, callables, means, pool, submit, finishes, job, biggest, advancement, came, learned, recording, exactly, best, way, across, parallel, tasks, temporary, manually, running, iterate, solarsystemids, correct, real, mean, script, reached, foxfour, seems, shouldn, seeing, timestamps, value, fetches, hour, performs, told, people, coded, around, xml, london, datacenter, notice, uptick, didn, bother, investigate, happily, schema, moving, tables, performed, against, discovered, occurring, day, errors, staled, major, deliverable, coming, figured, things, pain, points, optimize, wonderful, nature, creep, keeps, never, feel, progress, thepracticaldev, basically, map, inspired, http, www, machine9, net, 1111, helicityboson, outer, ring, universe, gurista, activity, northern, trade, hubs, 819, 421, 802, 640, without, snip, venal, tribute, forge, tenal, pure, blind, guristas, fade, deklein, k162, guides,

Text of the page (random words):
k162space c1 to c9 k162space c1 to c9 about k162 guides wormhole guide spotmarket graph sample posted 2016 06 07 author blake filed under nullsec pvp python spotmarket tags amarr branch deklein dodixie fade guristas jita pure blind rens tenal the forge tribute venal leave a comment what s data without some graphs here is a small snip of the graphing that is currently in the spotmarket project data jumps 2016 01 13 to 2016 06 07 17 802 640 rows kill 2016 01 13 to 2016 06 07 9 819 421 rows graphs jumps in trade hubs jumps and npc activity in the northern regions gurista ratting rates npc universe overview outer ring first pass at a map using d3 js inspired by helicityboson at http www machine9 net p 1111 spotmarket 0 6 posted 2016 06 07 author blake filed under python spotmarket 1 comment project update basically via thepracticaldev work on the spotmarket project has staled as i have a major work deliverable coming up at the end of the month i ve figured out some things have found pain points that i need to optimize and the wonderful nature of feature creep keeps adding more and more to the growing list of items that i need to work on so i never feel like i ve made any progress data errors after the xml api moved to the new london datacenter i started to notice a small uptick in ratting and kill rates i didn t bother to investigate and kept on happily coding as i was reworking the schema and moving data to different tables i performed a check against the dataset and discovered duplicate entries that were a second off occurring 3 4x a day i m using the cacheduntil value of the api as my timestamp with timestamp and solarsystemid as my primary key i have a process that fetches the data multiple times an hour and performs an import i ve been told by multiple people not to not use cacheduntil as the timestamp but this is what i ve coded the first time around i reached out to ccp foxfour and it seems that i shouldn t be seeing timestamps off by 1 second i have a temporary fix for this which is manually running a python process to iterate over all solarsystemids and correct duplicate data via a join query the real fix will mean reworking the consumer script parallel tasks the biggest advancement in my project came this month when i learned how to use python concurrent futures recording market prices at 0 8 records second isn t exactly the best way to scan 8 300 items across multiple regions python concurrent futures to lets you asynchronously execute callables this means that i can create a worker pool and submit jobs to it as each worker finishes it will get a new job here are 8 requests in my market scanner domain typeid 19 spodumain domain typeid 35 pyerite domain typeid 34 tritanium domain typeid 21 hedbergite domain typeid 22 arkonor domain typeid 18 plagioclase domain typeid 20 kernite domain typeid 36 mexallon what s so interesting about this check out the order the jobs were submitted in order typeid 19 20 etc and returned out of order because they finished at different times i scaled the import process up to 16 workers 8 cores x 2 and was able to scan 8 300 prices in about 30 minutes i have some work to do where i want to change the insert portion to only insert new data right now it is throwing the entire response into the database and relying on key constraints to dedupe characters the previous trading platform was good at showing transactions balances and item trends so i started to replicate a lot of the features for a first pass i wanted to start to integrate character information so i added api processes for wallet orders and blueprints i ll finish up with journal and assets in the next pass speculation i ve started to put my foot into the speculation market and built up a page to track my investments getting items into the table isn t elegant you have to throw a transactionid at an api endpoint so it records it from the wallet table into the speculation table i don t have enough jquery knowledge to create buttons to post to a api endpoint yet adding the ability to write back to the database will bring a lot of interaction to the project so it s high on the list empires of eve posted 2016 04 07 author blake filed under eveonline history nullsec 1 comment my copy of empires of eve arrived today so here are some unboxing shots if you can get the hardcover edition purchase it the finish quality and artwork really stands out spotmarket 0 4 posted 2016 03 18 author blake filed under python spotmarket tags amarr dodixie fountain jita plex rens leave a comment overview whew this was a big step in the right direction i added a lot of sanity keep everything in utc usability and cleanup to the existing design we ve got moongoo the beginnings of indexes and crest killmail data now flowing in to the growing dataset the backend was improved by migrating to postgresql 9 5 1 and i also cleaned up the install directions to try to make it easier 0 4 release bug change timestamp for zkillboard and markethistory consumer to use utc for mental sanity enhancement add 404 error page enhancement change to postgres 9 5 1 to support jsonb enhancement parameterize graph functions enhancement change date format in graphs to iso for mental sanity enhancement standardize table formatting enhancement supervisor to make flask web service persistent enhancement add jsonb index on data killmails for typeid and solarsystemid enhancement add paging to zkillboard consumer enhancement add check to resume from last recorded page for zkillboard consumer enhancement add basic exception handling to zkillboard consumer new creation of ship index report crest verified moon minerals it is no secret that i have an affinity for finding accurate moon data in a previous post i ve gone so far as to chart the regional density of each moon mineral showing that there were built in skews as to where each type was placed for each class r8 r16 etc in the rubicon expansion ccp introduced siphon units which can be anchored near a pos to slowly leech items from within the shields when an extractor is destroyed the killmail will show what moon mineral was in it this combined with the x y z location data that started to be exposed after the parallax expansion can be used to locate the nearest celestial object to give us a verified report of what moon mineral is contained in the object i gathered some examples for anyalisis and over a long flight i started to put some code together to use killmail x y z coordinates to parse crest killmails after implementation and testing it has proven to be accurate so i ve included the feature in this release there is a manual step to get the moon mineral data into a table where it is used in the web frontend i wanted to keep them separate to isolate the two datasets postgresql upgrade to 9 5 1 rather than writing a parser for crest killmails i decided to store the json itself for simplicity developing on 9 3 with the json datatype proved to be a path that i did not want to go down 9 3 is not the prefered version and lacked support for a fancy datatype inherit to 9 4 and above the following blog post also convinced me to migrate the proof is in the numbers so here s a side by side comparison of the same query on the same dataset with only the datatype being different the two virtual machines also had access to the same amount of cpu and ram to control the results this query was proof enough that i made the right choice later that night while checking my logging table i also noticed an improvement in insert speed 9 3 json kills insert 2982 93 81 rec sec 9 5 1 jsonb kills insert 2850 124 44 rec sec not bad for about 45 minutes of testing and correcting install instructions 0 5 release what s up for the next release check out the todo md for a full list of items slated for each release the main focus is going to be putting more control in the web front end letting you enable disable import items add item to the market zkillboard watch list etc spotmarket 0 3 dev posted 2016 03 02 author blake filed under market python spotmarket tags d3 js flask pandas r64 sansha 3 comments rapid affirmative coding the last four weeks have been a rather busy development cycle for me i finished up learning the basics of python have a working backend database and have become more conformable with the frontend side of my platform i ve restructured a lot of items and have a vision down for the final architecture the end goal is to have everything in a docker container so that anyone anywhere can deploy i want to aim for a push button receive data style of deployment let s keep the it wrangling to a minimum currently the frontend that i have is build on flask jinja2 as i have been developing the proof of concept out i ve found some limitations james who wrote the original trading program with me has been pushing me towards angularjs as the presentation framework luckily i ve kept to the original design principal of keeping every process that needs to communicate as an isolated component with this design any component can be replaced or moved and it is transparent to any other layer or neighboring service service a could can be replaced with no impact to the front or backend or the backend could be replaced with no impact to the services and so on laundry list of improvements and highlights 1 we ve got data 2 improved database inserts i rewrote the insert statements to use primary key constraints to perform error checking rather than selecting if the row exists and then inserting postgresql does have an upsert statement that i would like to utilize to gain even further insert speed but that will have to come when i update my postgresql version to 9 5 before kills insert 3128 49 85 rec sec after kills insert 2948 94 84 rec sec 3 restful api any presentation item that needs data receives it over an json api 4 multithreading flask app run host 0 0 0 0 port 80 debug true threaded true 5 using pgadmin for faster query generation 6 current state there is a lot of polishing needed but a lot of the bricks as i call them have been put in place learning the basics of d3 js how to return json from pandas manipulated postgresql data and bootstrap table for modern table ui all have been on a punch list to learn to get to this state what s next bring on a partner that can help guide the frontend deciding if flask jinja2 is really the right frontend platform for this project and me migrating graphing to use height length percentages rather than finite pixel amounts as the bootstrap style supports many different formats currently the charts look horrible on any tablet or mobile device parameterize charts and graphs write consumer for crest market prices write consumer for zkillboard crest killmails develop a proof of concept report for a region that takes only the regionid as an input so that i learn how to build reporting for any region or solar system leveraging the power of pandas by spending more time on the thinking problems developing indexes finding trends and learning about covariance reach a good polish so i can publish a 0 3 release learning how to create a docker image latest updates can be found on the 0 3 dev branch older entries k162space tweets by k162space recent posts spotmarket graph sample spotmarket 0 6 empires of eve spotmarket 0 4 spotmarket 0 3 dev categories aideron robotics dust514 eveonline failmail featured history industry market nullsec pos pvp python screenshot ships spotmarket uncategorized wormhole archives june 2016 april 2016 march 2016 february 2016 january 2016 december 2015 october 2015 january 2015 december 2014 november 2014 october 2014 august 2014 july 2014 june 2014 may 2014 april 2014 march 2014 february 2014 january 2014 december 2013 october 2013 august 2013 july 2013 june 2013 may 2013 april 2013 march 2013 february 2013 january 2013 december 2012 november 2012 october 2012 september 2012 august 2012 july 2012 june 2012 may 2012 april 2012 march 2012 february 2012 january 2012 december 2011 november 2011 october 2011 september 2011 august 2011 july 2011 june 2011 may 2011 april 2011 march 2011 february 2011 january 2011 december 2010 november 2010 october 2010 september 2010 august 2010 july 2010 june 2010 may 2010 april 2010 march 2010 february 2010 january 2010 december 2009 november 2009 october 2009 september 2009 august 2009 blogroll ardent defender eve fail eve manufacturing eve online wormholes eve opportunist eve prosper eve resurrection evehermit finders keepers inner sanctum of the ninveah jester s trek lovely little vagabond marketsforisk serpentine s eve the altruist tools chruker s eve database dotlan eve maps create a free website or blog at wordpress com mid mo design k162space create a free website or blog at wordpress com subscribe subscribed k162space join 92 other subscribers sign me up already have a wordpress com account log in now privacy k162space subscribe subscribed sign up log in report this content view site in reader manage subscriptions collapse this bar loading comments write a comment email required name required website