6 Commits

Author	SHA1	Message	Date
Marius Andra	1eeed28751	Fix Master EE code (#1701) ... * add test runner to ease pycharm dev * fix broken import * drop and recreate the clickhouse test db before running tests * fix person uuid str json serialization issue * make kafka optional in tests * fix inits * remove need for kafka in person.py * fix a bunch of mypy errors * fix function and add process_event to pipeline * fixed missing params and tests * change uuid and fix types * types * optimize for merge prop test * make ClickhouseProducer to produce to clickhouse one way or another * annotate types Co-authored-by: Eric <eeoneric@gmail.com> Co-authored-by: James Greenhill <fuziontech@gmail.com>	2020-09-24 06:14:17 -04:00
James Greenhill	dedf5582c1	first chunk of clickhouse framework (#1613) ... * first chunk of clickhouse framework * prod, not dev docker-compose.yml * add clickhouse sql files	2020-09-08 16:12:27 -07:00
James Greenhill	c2e03a3a46	Revert this to bring back the working copy of ee / clickhouse (#1588)	2020-09-04 21:12:07 -07:00
James Greenhill	5506135c3c	Bring back clickhouse changes along with queue size metrics (#1579) ... * Revert "Revert "Clickhouse setup (#1463)"" This reverts commit 7f2cab4b93. * add queue backlog to _stats endpoint * celery queue length into heartbeat -> statsd * reformat * type check fixes * bump expected number of queries to 8 Co-authored-by: Eric <eeoneric@gmail.com>	2020-09-04 13:44:53 -07:00
James Greenhill	7f2cab4b93	Revert "Clickhouse setup (#1463)" ... This reverts commit a0327587cb. Time to process events shot way up and logs are missing.	2020-09-03 19:27:02 -07:00
Eric Duong	a0327587cb	Clickhouse setup (#1463) ... * initial * migration command * migrations working * add modelless views for clickhouse * initial testing structure * use test factory * scaffold for all tests * add insight and person api * add basic readme * add client * change how migrations are run * add base tables * ingesting events * restore delay * remove print * updated testing flow * changed sessions tests * update tests * reorganized sql * parametrize strings * element list query * change to seralizer * add values endpoint * retrieve with filter * pruned code to prepare for staged merge * working ingestion again * tests for ee * undo unneeded tests right now * fix linting * more typing errors * fix tests * add clickhouse image to workflow * move to right job * remove django_clickhouse * return database url * run super * remove keepdb * reordered calls * fix type * fractional seconds * fix type error * add checks * remove retention sql * fix tests * add property storage and tests * merge master * fix tests * fix tests * . * remove keepdb * format python files * update CI env vars * Override defaults and insecure tests * Update how ClickHouse database gets evaluated * remove bootstrapping clickhouse database routine * Don't initialize the clickhouse connection unless we say it's primary * . * fixed id generation * remove dump * black settings * empty client * add param * move docker-compose for ch to ee dir * Add _public_ key to repo for verifying self signed cert on server * update ee compose file for ee dir * fix a few issues with tls in migrations * update migrations to be flexible about storage profile and engine * black settings * add elements prop tables Co-authored-by: James Greenhill <jams@uber.com>	2020-09-03 10:27:45 -07:00