* ci: Run core backend tests with both HogQL and legacy insights
* Double the number of tests
* Fix env var setting
* Add some HOGQL_INSIGHTS_OVERRIDE overrides in tests
* Mark `Insight.last_refresh` as deprecated
* Fix bad merge
* Update query snapshots
* Update query snapshots
* Update test_team.py
* Actually remove legacy backend from matrix
* Update test_fetch_from_cache.py
* Use `update_cached_state` in `calculate_for_query_based_insight()`
* Clean up CI changes and a comment
* Fix `update_cached_state` typing
* Update test_fetch_from_cache.py
* Update test_insight_cache.py
* Update test_insight_cache.py
* Clarify `generate_insight_cache_key` as legacy function
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Revert "chore(CI): revert "chore: syphar is deprecated. fangle python actions…"
This reverts commit 1f2ff4ac32.
* not on depot?
* was missing a syphar removal :/
* was missing a syphar removal :/
* Update query snapshots
* back to depot
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Added suppoer for joined tables and using autocomplete on join constraints
* Added a type for Expression fields
* Fixed mypy
* Fixed mypy
* Update query snapshots
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Remove old dockerfile and only build a single image with unit
We have migrated over to the `unit` image (instead of gunicorn) but we
were still building both. This could lead to confusion - we have no
plans to move back so let's remove the second image build and only
build the unit based image
* Remove unit build id
* fix: Checkout master before checking for hogql changes
* fix: But go back to branch after done
* fix: Instead just diff with origin/master
* fix: Install libantlr, what's the worst that can happen?
* fix: Install antlr the hogql-way
* fix: Let's just checkout everything
* feat(batch-exports): Add backfill model and service support
* test(batch-exports-backfills): Add Workflow test
* chore(batch-exports-backfill): Bump migration
* feat(batch-exports): Add RedshiftBatchExportWorkflow
* feat(batch-exports): Add Redshift to BatchExport destinations
* feat(batch-exports): Support properties_data_type Redshift plugin parameter
* refactor(batch-exports): Insert rows instead of using COPY
* fix: Remove unused migration
* chore: Require aiokafka
* feat: Implement new structlog batch exports logger
* refactor: Use new structlog logger everywhere
* test: Add tests, fix things
* fix: Remove old tests
* chore: Change typing of return logger
* chore: Bump structlog
* chore: Extend docstrings
* fix: Don't use async logging as it's unsupported by temporal runtime
* test: Add logger tests
* fix: Mix pytestmark lists
* fix: Remove unused imports
* fix: Cleanup pytest warnings
* fix: Create and drop dataset for bigquery tests
* fix: Typing issue?
* fix: Let's just checkout everything
* fix: Use global event loop in tests
* fix: Blow-up cache
* fix: Truncate only if table exists
* revert-me: Skip all postgres tests
* fix: Connect to kafka in localhost
* fix: Lazily connect to Kafka
* fix: Resolve conflicts
* fix: Capture temporal context once and bind it to the logger
* fix: Make configure logger sync
* fix: Keep strong reference to background tasks
* fix: Continue consuming from log queue even if we fail to produce
* fix: Also catch the producer not starting
* fix: Remove unused await
* fix: Log kafka producer error after logger is configured
* fix: Replace deprecated set-output GH actions command
* fix: Checkout master before checking for hogql changes
* fix: But go back to branch after done
* fix: Instead just diff with origin/master
* fix: Let's just fetch manually
* fix: Install libantlr, what's the worst that can happen?
* fix: Install antlr the hogql-way
* fix: Amazing hogql-developers already include this in wheel
* Allow placeholders in place of ratio expressions in hogql
* Added C++ changes for RatioExpr
* Bump `hogql_parser` version
* Upgrade ANTLR version to align with brew latest
* Update query snapshots
* Fix parser change message body
The quotes weren't right, so the step failed instead of posting a comment.
* Revert "Update query snapshots"
This reverts commit 1302d3b24f.
* Update MD5 sum
* Bump version again
* Use new hogql-parser version
---------
Co-authored-by: Michael Matloka <michal@matloka.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Matloka <dev@twixes.com>
* Add partial C++ parser of HogQL
* Support all the rules and add proper error handling
* Use `AlignAfterOpenBracket: BlockIndent`
* Reuse all the parser tests for the C++ backend
* Fix relationship between headers and implementations
* Add more error handling and fix minor issues
* Build both Python and C++ parsers in the package.json script
* Update ARRAY JOIN error assertion
* Improve timeit
* Move the C extension to the top level
* Refactor `vector_to_list_string`
* Build the parser on Linux
* Build wheels for the parser
* Simplify Linux build and fix macOS
* Fix Homebrew paths on x86 and don't fail fast in CI
* Set MACOSX_DEPLOYMENT_TARGET for C++20
* Set up QEMU for Linux ARM builds
* Publish the wheels on PyPI
* Avoiding Linux ARM emulation in CI for now
* Build sdist too
* Revert Dockerfile changes
* Fix PyPI publish
* Add README and optimize sdist build
* Use setup.py directly instead of build
* Use PyPI hogql-parser instead of local
* Also revert production-unit.Dockerfile
* Fix sdist upload and add Linux ARM back
* No Linux ARM build in the end
* Fix artifact uploading
* Do try building Linux ARM
We need this for prod.
* Use `npm` in `grammar:build`
`pnpm` is not available in that job.
* Fix formatting of hogql_parser
* Build everything on macOS
* Revert "Build everything on macOS"
Not so fast actually.
* Use hogql-parser=0.1.1
* Fix dylib in macOS wheel
* Bump hogql-parser version
* Fix missing module error
* Delete timeit.py
* Make error handling robust
* Format the C++
* Use `hogql-parser==0.1.1`
* Fix reserved keyword error assertions
* Use HEAD hogql_paresr in CI
* Fix `apt` usage
* Add some sudo in CI
* Ensure package will be releasable before build
* Bump version to 0.1.3
* Cover C++ `unquote_string` with tests
* Use BuildJet ARM runners for ARM builds
* Add some instructions
* Add HogQL version check to backend CI
* Update requirements.txt
* Use `setuptools` instead of the deprecated `distutils`
* Fix working dir in backend CI
* Align ANTLR versions
* Add test for "mismatched input"
This is thrown differently than other HogQLSyntaxExceptions in C++, so might help reveal what's going on with tests failing only on Linux CI and not macOS dev
* Add types and bump version
* Comment instead of failing version check
* Automate hogql-release version bump
* Fix checkout token
* Don't build hogql-parser if there were no changes
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Improve documentation
* Use new hogql-parser version
* Fix error start and end initialization
* Note `antlr4-cpp-runtime`
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Also remove NUL chars in C++
* Check ANTLR4 runtime archive checksum for security
* Note more decrefs to add
* Add vector size checks
* Use new hogql-parser version
* Don't support the `start` arg in C++ `parse_expr`
* Use new hogql-parser version
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Add docker image that uses nginx unit instead of gunicorn
🦄🔫
* Add unit build to CI
* Fix duplicate id
* try 3.11
* Only build for amd64
need python3.11 for unit image on arm
* ci-backend runs-on -> buildkite
* Use buildjet runners
* Go away old cache
* Fix deps
* Ensure to update the package registry
* Use bigger runners for django tests
* Use ARM for jobs with less concurrency
* Revert "Use ARM for jobs with less concurrency"
This reverts commit bcbf0d7524.
* Fix typo
