Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(experiments): Apply correct filters to "View recordings" #27849

Merged
merged 5 commits into from
Jan 24, 2025

Conversation

danielbachhuber
Copy link
Contributor

See https://posthog.slack.com/archives/C087ZBJBZ7C/p1737128365755459

Also reported in a direct email:

I’ve been running another no-code experiment on PostHog. The recordings show that not everyone is seeing the variant I set, even when I filter based on the experiment and variant. For example, if my control is ABC and the variant is CBA, the recordings for CBA sometimes show the control instead. This makes me question the accuracy of the stats... Is this a bug or user error on my part?

Changes

Introduces a getViewRecordingFilters() function with tests to generate the various permutations of filters based on the metric. Note: the screencasts are solely to show the filters being applied; I don't actually have any recordings because I generated the events synthetically.

If no filters could be identified, then the button will be disabled:

CleanShot 2025-01-23 at 15 48 16@2x

Funnel with events

CleanShot.2025-01-23.at.15.48.50.mp4

Funnel with action

CleanShot.2025-01-23.at.15.49.42.mp4

Trend with default exposure

CleanShot.2025-01-23.at.15.52.18.mp4

How did you test this code?

Tests should pass + manual review.

@danielbachhuber danielbachhuber requested a review from a team January 23, 2025 23:52
Copy link
Contributor

Size Change: -5 B (0%)

Total Size: 1.16 MB

ℹ️ View Unchanged
Filename Size Change
frontend/dist/toolbar.js 1.16 MB -5 B (0%)

compressed-size-action

Copy link
Contributor

@andehen andehen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one!

Never tried this button myself for the same reasons (no recordings locally). But it is a good reminder that we should dog food ourselves more and run actual experiments ourselves. Even dummy ones would be good, just to try out the functionality in a more realistic setting.

@danielbachhuber danielbachhuber merged commit 9c4d9e8 into master Jan 24, 2025
99 checks passed
@danielbachhuber danielbachhuber deleted the experiments/fix-view-recordings branch January 24, 2025 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants