Skip to content

Commit d0fedf2

Browse files
committed
Updates MMM comparison and adds benchmark
Refines the comparison of PyMC-Marketing with other MMM packages, highlighting feature differences and standardizing terminology. Adds a detailed performance benchmark against Google Meridian, showcasing PyMC-Marketing's superior speed, accuracy, and scalability. Includes updated recommendations for choosing the appropriate MMM library based on user needs and requirements.
1 parent 75646e2 commit d0fedf2

File tree

2 files changed

+19
-12
lines changed

2 files changed

+19
-12
lines changed
Lines changed: 19 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# How We Compare
22

3-
Given the popularity of the Media Mix Modelling (MMM) approach, numerous packages are available. Below is a concise comparison highlighting how PyMC-Marketing stands against other popular options:
3+
Given the popularity of the Media Mix Modelling (MMM) approach, numerous packages are available. Below is a concise comparison highlighting how the features of PyMC-Marketing stands against other popular options:
44

55
| Feature | PyMC-Marketing | Robyn | Orbit KTR | Meridian* |
66
|-------------------------------|:--------------:|:--------------:|:---------:|:----------------------:|
@@ -9,29 +9,32 @@ Given the popularity of the Media Mix Modelling (MMM) approach, numerous package
99
| **Foundation** | PyMC | - | STAN/Pyro | TensorFlow Probability |
1010
| **Company** | PyMC Labs | Meta | Uber | Google |
1111
| **Open source** |||||
12-
| **Model Building** | 🏗️ Build | 🏗️ Build | 🏗️ Build | 🏗️ Build |
1312
| **Out-of-Sample Forecasting** |||||
1413
| **Budget Optimizer** |||||
1514
| **Time-Varying Intercept** |||||
1615
| **Time-Varying Coefficients** |||||
17-
| **Custom Priors** || |||
18-
| **Custom Model Terms** | | || |
16+
| **Custom Priors** || NA |||
17+
| **Custom Model Terms** | | |||
1918
| **Lift-Test Calibration** |||||
20-
| **Geographic Modeling** |||||
19+
| **Hierachical Geographic Modeling** |||||
20+
| **Standardized Database Connectors** | ✅ (with Fivetran) ||| ✅ (limited to Google ecosystem) |
2121
| **Unit-Tested** |||||
22-
| **MLFlow Integration** |||||
23-
| **GPU Sampling Accelleration**|| - |||
24-
| **Consulting Support** | Provided by Authors | Third-party agency | Third-party agency | Third-party agency |
22+
| **MLFlow Integration** |||||
23+
| **Multiple Sampling Backends**|| NA |||
24+
| **GPU Sampling Acceleration**|| NA |||
25+
| **Consulting Support** | Provided by Authors | Third-party agency | Third-party agency | Third-party agency |
2526

2627
*\*Meridian has been released as successor of Lightweight-MMM, which has been deprecated by Google*
2728

28-
Last updated: 2025-08-07
29+
Last updated: 2025-10-17
2930

3031
---
3132
### Key Takeaway
3233
Four of the five major libraries for MMM models implement different flavors of Bayesian models. While they share a broadly similar statistical foundation, they differ in API flexibility, underlying technology stack, and implementation approach.
3334

34-
PyMC-Marketing stands out as the most widely used library by PyPI downloads (see plot below), offering unmatched flexibility and a comprehensive set of advanced features. This makes it ideal for teams looking for a highly customizable, state-of-the-art solution. However, its breadth and depth also make it the most sophisticated option, which may require a steeper learning curve. Other libraries have their own strengths—for example, Google Meridian features a more opinionated API and seamless integration with the Google ecosystem, which can be advantageous for organizations already embedded in Google's stack.
35+
PyMC-Marketing stands out as the most widely used library by PyPI downloads (see plot below), offering unmatched flexibility and a comprehensive set of advanced features. This makes it ideal for teams looking for a highly customizable, state-of-the-art solution.
36+
37+
However, its breadth and depth also make it the most sophisticated option, which may require a steeper learning curve. Other libraries have their own strengths — for example, Robyn is popular among the R community and provides extensive tutorials and documentation.
3538

3639
Your optimal choice should depend primarily on:
3740

@@ -41,6 +44,10 @@ Your optimal choice should depend primarily on:
4144

4245
![MMM Downloads Analysis](./mmm_downloads_analysis.png)
4346

47+
## Detailed Performance Benchmark
48+
49+
When it comes to Bayesian Media Mix Modeling the two most used options are PyMC-Marketing and Google Meridian. Our comprehensive technical benchmark comparing PyMC-Marketing against Google Meridian across realistic datasets (from startup to enterprise scale) reveals PyMC-Marketing's superior performance: **2-20x faster sampling**, **40% lower error** in channel contribution estimates, and **successful scaling** to large enterprise datasets where Meridian fails to converge. PyMC-Marketing's flexible sampling backends (NumPyro, BlackJAX, Nutpie) provide significant advantages over Meridian's fixed TensorFlow Probability implementation. See our [detailed benchmark analysis](https://www.pymc-labs.com/blog-posts/pymc-marketing-vs-google-meridian) for complete results and open-source methodology.
50+
4451
## Our Recommendation
4552

4653
### Choose Meta Robyn if:
@@ -53,12 +60,12 @@ Your optimal choice should depend primarily on:
5360

5461
- You want a simplified (albeit less flexible) API to build models across geographies
5562
- Direct integration with the Google advertising ecosystem is important
56-
- You want strong integration with other Google products such as Collab
63+
- You can allow for reduced predictive accuracy and explainability
5764

5865
### Choose PyMC-Marketing if:
5966

6067
- Maximum flexibility for complex, unique business requirements is necessary
6168
- You need advanced statistical modeling capabilities (e.g., Gaussian Processes)
6269
- Production ready setup and integration into broader data science workflows is important (MLflow)
6370
- You prefer independence from major ad publishers and networks
64-
- Professional consulting support is desirable
71+
- Professional indipendent consulting support is desirable info@pymc-labs.com
145 KB
Loading

0 commit comments

Comments
 (0)