Optimizely Experimentation is the umbrella for Web Experimentation, Feature Experimentation (formerly Full Stack), Personalization and related analytics. It is licensed on usage, and two metrics matter: monthly active users (MAUs) and impressions. The Optimizely Data Platform (ODP) shares the MAU label but defines it differently. The Usage Metrics define each unit, and support articles explain the mechanics.[1]
Editions
| Product | Catalog row | Main metric |
|---|---|---|
| Web Experimentation (snippet-based) | Optimizely Web Experimentation | Monthly Active Users or Impressions per Year |
| Feature Experimentation (SDK-based flags and experiments) | Optimizely Feature Experimentation | Monthly Active Users or Impressions per Year |
| Personalization | Optimizely Personalization | Impressions per Year, Pageviews per Year |
| Optimizely Data Platform | Optimizely Data Platform | Monthly Active Users (profiles) |
| Analytics (warehouse-native) | Optimizely Analytics | Events |
Which metric applies is set in the Order. The documentation says that starting in September 2020 Optimizely Experimentation introduced MAUs as a simplified usage billing component, and customers billed by impressions view them in a separate usage summary.[4] Legacy Full Stack projects, created before February 2021, follow separate impression rules.[4]
Three product terms are specific to this area. The Product Use Terms state that Experimentation, Personalization and Analytics are US-hosted only, with EU hosting available for visitor data from 2025-03-01 while administrative data stays in the US and raw event exports are unavailable for EU-hosted customers until further notice.[2] For ODP, the customer receives a limited, revocable, non-exclusive, non-transferable, royalty-free licence to install the Website Tag solely for ODP use during the term, and Audience Sync follows the Experimentation or CMS documentation and excludes the ODP user interface.[2] Analytics connects to the customer’s warehouse through a customer-provisioned service account limited to customer-selected tables, sends encrypted SQL queries and does not process data outside the warehouse, modify warehouse data or use it for training.[2]
Metrics
Monthly active users
The contract definition is split by product.[1]
- Feature or Full Stack Experimentation: the total number of unique user IDs that appear in all calls to Optimizely’s SDKs or APIs per month.
- Web Experimentation: the total number of unique user IDs evaluated by the snippet per month, even if the user is not participating in an experiment.
- Optimizely Data Platform and Content Recommendations and Intelligence: the total number of profiles with an active event from any source collected per month. An active event is a tracking signal containing a user identifier, either a customer action (also called an event) or a customer update.
- Traffic linked to the same user through a unique ID counts once across Web and Feature Experimentation.
Support documentation adds detail. In Feature Experimentation an MAU is counted once per month for each unique user ID that triggers at least one decision or tracking event; decision events for disabled flags and targeted delivery rules are not currently counted.[3] In Web Experimentation, every unique visitor who encounters a page where the snippet is loaded is counted, even if the snippet is empty and contains no experiments, so the way to avoid counting is to remove or disable the snippet on pages where Optimizely should not run.[3] An MAU is de-duplicated across projects when the user ID is the same, and customers using both products can override anonymous Web IDs with known Feature Experimentation IDs through “bring your own visitor ID” to avoid overcounting.[3]
MAUs are calculated on the server timestamp, so they can be verified down to experiment level with the Experimentation Events Export, which lists all MAUs in a period for comparison with the invoice.[3] The same article states that MAUs are a shared pool between Web and Feature Experimentation and roll over to the next month if not used, which the contract definition does not itself state, so the Order should confirm it.[3]
Orders signed under the 2021 definitions differ on one point. They state that when both Full Stack and Web Experimentation are used, total MAUs equal the sum of the MAUs calculated separately for Full Stack and Web.[6] The current Usage Metrics instead count a unique user ID once across the two.[1] A customer on an older order should not assume the newer rule. See Monthly Active Users.
Impressions
For Web Experimentation, impressions are counted each time an experiment or variation is activated on a Page, where a Page is a section of a webpage chosen for personalisation or experimentation, which can be a whole page or specific elements. Multiple experiments on one Page, or one experiment on several Pages such as a header element, produce multiple impressions.[1] For Feature or Full Stack, impressions are counted each time an experiment or variation delivers a decision event through the Event API showing the user is in an experiment. For both, decision events are de-duplicated over a fixed five-second window. For Product Recommendations, an impression is each set of recommendations, counted as one widget or container, returned to a delivery point through an API call.[1]
The worked example in support documentation shows why impressions can grow quickly: a homepage with three experiments and one global page counts four impressions per visit, and a refresh adds four more; three searches in a Feature Experimentation search experiment add three, for eleven in total, whereas the same visitor is one MAU.[4] Visitors bucketed into a holdback do not see the experiment and generate no impressions.[4] Impressions are processed when sent, so a delayed pipeline can put old impressions on a current bill without distinguishing them.[4] In legacy Full Stack projects, impressions are counted only for visitors bucketed into a variation, and all rollouts are excluded.[4] See Impressions per Year.
Events and pageviews
For Analytics and warehouse-native experimentation analytics, an Event is a row in an event table in the customer’s data warehouse that is exposed to the service, and the monthly number is the average across all event tables over the past three months.[1] See Events. Pageviews, where used, count each view of a customer page that uses content provided by the service, and a reload is an additional Pageview.[1] See Pageviews per Year.
Counting / floors
Usage Volumes are set per Order. The Usage Metrics define overage as the number of impressions generated above the agreed IPY, or the number of MAUs per Contract Year above the agreed volume.[1] Overage Fees are two times the Usage Volume unit price, accrue from the date the overage first occurs and are invoiced monthly in arrears.[7] The customer is to monitor its own use and Optimizely may review it at any time.[7]
Forecasting guidance in the documentation asks where Experimentation will run, how many unique users or visitors exist per month excluding bot traffic, how complete the implementation is (90 to 100 percent if the SDK or snippet runs for most users, 5 to 10 percent for a few landing pages) and expected annual growth.[3] Because the Web snippet counts every visitor on a page where it loads, the implementation share is closer to the share of pages carrying the snippet than the share of pages with live experiments.
Virtualization & partitioning
There is no installed component to partition. Two structural points matter. Experimentation usage is measured across projects in an account and de-duplicated by user ID, so splitting traffic across multiple projects does not multiply MAUs for the same ID.[3] And a user whose ID is anonymous in Web and known in Feature Experimentation may be counted as two MAUs unless the IDs are aligned.[3]
Cloud / BYOL
Experimentation is delivered as a hosted service, with Edge Delivery defined as an SDK for Web Experimentation that is an alternative to the JavaScript snippet.[2] The customer’s own data warehouse is the data source for Analytics; Optimizely states that it does not process data outside the warehouse.[2] No bring-your-own-licence rights are described.
Programs
- Usage dashboards. Administrators with the Opti ID Super Admin or Billing Admin role can see MAUs used and remaining, projected usage, cumulative MAU usage, and MAU usage by experiment and by project, updated daily.[5]
- Billing notifications. Experimentation notifications are sent at 25%, 50%, 75% and 100% of allotted usage and at every ten percent beyond 100%; they do not prevent overage.[5]
- Free Access and beta. Trials and pre-production features are outside GTC warranties and support.[2] See Free Access and beta releases.
- Onboarding and catalogue services for Experimentation and Personalization are pre-defined and pre-paid, with unused hours expiring.[2] See Onboarding hours.
Audits and compliance
The MAU or impression count is calculated by Optimizely, so the audit surface is reconciliation. Compare the Plan tab in Account Settings (Web or Feature Experimentation administrators) or the Opti ID dashboards with the Events Export, and the invoice with both.[3] Review which pages load the Web snippet, which SDK calls pass a user ID that is not an experiment participant, and whether Product Recommendations widgets multiply impressions. Overage notifications from Optimizely are written (email acceptable), and overage accrues from the first day.[7]
Out of scope
This article does not cover CMS, Commerce, CMP, Opal or Campaign, the pricing of any Order, or the detailed behaviour of individual SDK methods beyond the counting rules cited.