Variance reduction

[s3alfisc]

• Add a tutorial section
• Write notebook on variance reduction with pyfixest

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
core-tests	76.78% <ø> (ø)
tests-extended	?
tests-vs-r	15.81% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[apoorvalal]

btw one way to distinguish from CUPED might be to allow for (1) heterogeneity in the autoregressive parameter in the DGP, which I remember messing with when I originally wrote this DGP to test things out and (2) bigger variance in time FEs, which CUPED averages over but 2WFE captures correctly.

[s3alfisc]

Nice, I'll play around with it & add a section =)

[Dpananos]

Small typo

Because linear regression is just a fency way to compute and compare differences, we could also have estimated this via pf.feols():

Should be

Because linear regression is just a fancy way to compute and compare differences, we could also have estimated this via pf.feols():

[s3alfisc]

Oh sweet, thank you!

[s3alfisc]

Any other thoughts on the vignette? @apoorvalal suggested to compare CUPED with panel regressions when there are auto correlated errors, as panel regression should look better in this case?

[Dpananos]

One for me below, curious what @juanitorduz thinks since I think this PR is related to #1017?

The example shown has 30 observations per subject. I think that would correspond to a case where you randomize subjects over T days, and then only analyze the experiment after T+30 days (waiting for the subjects on the last day of randomization to complete their 30 day observation period).

In reality, we may want to do inference before all subjects have their 30 day observation period (depends on the design I suppose). In the case where not all subjects have the same number of observations, we would need to use a ratio metric + the delta method to get the right pointe estimate and standard error. I don't think there is (yet) a CUPED interpretation for ratio metrics as a subject level outcome regression which makes it hard to use in this case.

My understanding is that we could instead use a two way fixed effect (or some other approach, I'm not completely sure).

Do you think it might be worthwhile to explore a case in which the experiment does not have the same observation length for all subjects?