5 Commits

Author	SHA1	Message	Date
James Greenhill	adeb7694cc	Publish events to Kafka for consumption (#1644) ... * Publish events to Kafka for consumption * Commit avro idl's for event schemas * convert client to use github.com/dpkp/kafka-python * events loaded into clickhouse from Kafka * remove cruft * Publish events to Kafka for consumption * convert client to use github.com/dpkp/kafka-python * remove cruft * include kafka migrations * bugfixes for migrations * use constants for consistency * wrap up local migrations * small fixes * tune ups	2020-09-15 20:04:38 -07:00
James Greenhill	c2e03a3a46	Revert this to bring back the working copy of ee / clickhouse (#1588)	2020-09-04 21:12:07 -07:00
James Greenhill	5506135c3c	Bring back clickhouse changes along with queue size metrics (#1579) ... * Revert "Revert "Clickhouse setup (#1463)"" This reverts commit 7f2cab4b93. * add queue backlog to _stats endpoint * celery queue length into heartbeat -> statsd * reformat * type check fixes * bump expected number of queries to 8 Co-authored-by: Eric <eeoneric@gmail.com>	2020-09-04 13:44:53 -07:00
James Greenhill	7f2cab4b93	Revert "Clickhouse setup (#1463)" ... This reverts commit a0327587cb. Time to process events shot way up and logs are missing.	2020-09-03 19:27:02 -07:00
Eric Duong	a0327587cb	Clickhouse setup (#1463) ... * initial * migration command * migrations working * add modelless views for clickhouse * initial testing structure * use test factory * scaffold for all tests * add insight and person api * add basic readme * add client * change how migrations are run * add base tables * ingesting events * restore delay * remove print * updated testing flow * changed sessions tests * update tests * reorganized sql * parametrize strings * element list query * change to seralizer * add values endpoint * retrieve with filter * pruned code to prepare for staged merge * working ingestion again * tests for ee * undo unneeded tests right now * fix linting * more typing errors * fix tests * add clickhouse image to workflow * move to right job * remove django_clickhouse * return database url * run super * remove keepdb * reordered calls * fix type * fractional seconds * fix type error * add checks * remove retention sql * fix tests * add property storage and tests * merge master * fix tests * fix tests * . * remove keepdb * format python files * update CI env vars * Override defaults and insecure tests * Update how ClickHouse database gets evaluated * remove bootstrapping clickhouse database routine * Don't initialize the clickhouse connection unless we say it's primary * . * fixed id generation * remove dump * black settings * empty client * add param * move docker-compose for ch to ee dir * Add _public_ key to repo for verifying self signed cert on server * update ee compose file for ee dir * fix a few issues with tls in migrations * update migrations to be flexible about storage profile and engine * black settings * add elements prop tables Co-authored-by: James Greenhill <jams@uber.com>	2020-09-03 10:27:45 -07:00