GSC Portfolio Analysis: Run Multiple SEO Experiments, Get One Verdict

Running one SEO experiment is useful. Running ten at once and understanding the portfolio-level impact is transformative. GSC Portfolio Analysis takes your Google Search Console experiment data (multiple pages, control and treatment groups, impressions, clicks, CTR, and position) and delivers batch verdicts, effect size distributions, and projected click impact across your entire experiment portfolio. Upload your CSV and see which experiments won, which lost, and what the aggregate impact looks like.

Worked Example: Did the Test Group Get a Higher Click Rate?

Question. Was the click rate higher for the test group in this sample?

Inputs. Four page tests use made-up data (simulated, seed 23). Each has clicks and views for a control group and a test group. The script draws clicks at set rates.

Built in. Control rates are 4.0, 3.5, 5.0, and 3.0 percent. Test rates are 4.4, 3.2, 5.4, and 3.3 percent. So three of the four tests have a small rise built in, and one (title_b) has a fall. The pooled gap built in is about +0.23 percentage points.

Method. Add the clicks and views for each group. Divide clicks by views. Compare the two pooled rates, and give an approximate 95% interval for the gap, using a normal approximation applied to the two pooled groups. It does not model the four tests separately, so the interval is illustrative for this made-up sample, not a per-experiment estimate.

import numpy as np
from math import sqrt

# Simulated data, seed 23
rng = np.random.default_rng(23)
experiments = [
    ("title_a", 1200, 0.040, 0.044),
    ("title_b", 950, 0.035, 0.032),
    ("meta_c", 1500, 0.050, 0.054),
    ("content_d", 800, 0.030, 0.033),
]
rows = []
for name, n, p_control, p_treatment in experiments:
    control_clicks = int(rng.binomial(n, p_control))
    treatment_clicks = int(rng.binomial(n, p_treatment))
    rows.append((name, n, control_clicks, n, treatment_clicks))

control_impressions = sum(row[1] for row in rows)
control_clicks = sum(row[2] for row in rows)
treatment_impressions = sum(row[3] for row in rows)
treatment_clicks = sum(row[4] for row in rows)
control_ctr = control_clicks / control_impressions
treatment_ctr = treatment_clicks / treatment_impressions
lift_pp = (treatment_ctr - control_ctr) * 100
relative_lift = (treatment_ctr / control_ctr - 1) * 100

print("Simulated data, seed 23")
print("Experiment | Control clicks/impressions | Treatment clicks/impressions")
for name, ci, cc, ti, tc in rows:
    print(f"{name} | {cc}/{ci} | {tc}/{ti}")
print(f"Portfolio control: {control_clicks}/{control_impressions} "
      f"= {control_ctr:.4%}")
print(f"Portfolio treatment: {treatment_clicks}/{treatment_impressions} "
      f"= {treatment_ctr:.4%}")
print(f"Difference: {lift_pp:+.3f} percentage points")
print(f"Relative change: {relative_lift:+.2f}%")
se = sqrt(control_ctr * (1 - control_ctr) / control_impressions
          + treatment_ctr * (1 - treatment_ctr) / treatment_impressions)
gap = treatment_ctr - control_ctr
lo = (gap - 1.96 * se) * 100
hi = (gap + 1.96 * se) * 100
print(f"Approximate 95% interval for the difference: {lo:+.3f} to {hi:+.3f} "
      "percentage points")
print("Built in: control rates 4.0, 3.5, 5.0, 3.0 percent; treatment rates "
      "4.4, 3.2, 5.4, 3.3 percent (pooled gap about +0.23 points)")

Printed output:

Simulated data, seed 23
Experiment | Control clicks/impressions | Treatment clicks/impressions
title_a | 50/1200 | 55/1200
title_b | 39/950 | 25/950
meta_c | 70/1500 | 84/1500
content_d | 24/800 | 29/800
Portfolio control: 183/4450 = 4.1124%
Portfolio treatment: 193/4450 = 4.3371%
Difference: +0.225 percentage points
Relative change: +5.46%
Approximate 95% interval for the difference: -0.611 to +1.061 percentage points
Built in: control rates 4.0, 3.5, 5.0, 3.0 percent; treatment rates 4.4, 3.2, 5.4, 3.3 percent (pooled gap about +0.23 points)

Result. The control group had 183 clicks out of 4450 views (4.1124%). The test group had 193 clicks out of 4450 views (4.3371%). The gap was +0.225 percentage points, a +5.46% rise from the control rate, close to the +0.23 built in. One of the four tests goes the other way: title_b fell from 39 clicks to 25, and the pooled figure averages that fall with the three rises. The approximate 95% interval for the pooled gap runs from -0.611 to +1.061 percentage points, so it includes zero. With 4450 views per group, the interval is wide and includes zero, so this sample cannot tell a small real rise from chance.

Limit. This small made-up example does not show that a page change caused the gap. The interval is approximate, and the pooled number hides the differences between tests.

Sources. Wilson EB. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 1927. doi.org/10.1080/01621459.1927.10502953. Wilson published the confidence interval for a proportion in 1927. The interval in the example above is a simpler normal approximation, not the Wilson interval.

What Is GSC Portfolio Analysis?

When you run SEO experiments (changing title tags, meta descriptions, or page content) you typically analyze each experiment individually. Did the new title increase CTR? Did the meta description change affect impressions? One-at-a-time analysis works for a single test, but if you are running experiments across your blog, your product pages, and your landing pages simultaneously, you need a portfolio view that answers bigger questions: What is my overall win rate? Are certain types of changes more effective? What is the cumulative click impact of all my experiments?

GSC Portfolio Analysis treats your experiments as a managed portfolio, similar to how an investment manager evaluates a portfolio of trades rather than individual positions. Each experiment gets a verdict (win, loss, or inconclusive) based on statistical significance and effect size. Then the portfolio is analyzed as a whole: overall win rate, average effect size, distribution of outcomes, and projected additional clicks.

This is the analysis that moves SEO experimentation from ad hoc to systematic. Instead of running experiments and hoping for the best, you can see your hit rate and project the value of your next batch of tests.

When to Use GSC Portfolio Analysis

The ideal time to run this analysis is after a batch of SEO experiments has completed, once Search Console has collected enough data to judge outcomes. If you changed title tags on 15 pages three weeks ago, export the before-and-after GSC data for those pages and run the portfolio analysis.

Monthly experiment reviews are a natural cadence. If your team runs a steady stream of title tests, meta description experiments, or content updates, the portfolio analysis gives you the monthly scorecard: how many experiments ran, how many won, what the aggregate impact was on clicks and impressions, and what to try next month.

Strategic planning also benefits from this analysis. If you are debating whether to invest more in SEO experimentation, the portfolio analysis estimates the click impact. A portfolio summary with a win rate and an average lift per winning experiment gives a projectable click gain to weigh against the cost of continued investment, which you supply.

Cross-functional alignment is another important trigger. SEO teams often struggle to communicate results to stakeholders who do not understand individual test metrics. A portfolio summary with an overall win rate, aggregate click gains, and projected annual impact translates SEO experimentation into language that executives and budget holders understand. Instead of "we improved CTR by 0.8 percentage points on one article," you can report a summary of all tests: wins and estimated click gains. Show the data and how the total was worked out, so readers can judge the result.

What Data Do You Need?

You need a CSV with one row per experiment observation. The eight required columns are: exp_id (a unique identifier for each experiment), group (control or treatment), page_url (the page being tested), search_impressions, search_clicks, click_rate (CTR as a decimal), serp_position (average ranking position), and adjusted_ctr (position-adjusted CTR that accounts for ranking changes). Two optional columns add depth: query (the search query, enabling query-level analysis) and period_date (date stamps, enabling time-series views).

The simplest way to build this dataset: for each experiment, export GSC page-level data for the control period (before the change) and the treatment period (after the change). Add an exp_id column to identify the experiment and a group column to label control vs. treatment rows. Stack all experiments into one CSV.

Parameters include batch_assignments (default "auto": automatically groups experiments into batches by timing), content_type_regex (classifies pages by URL pattern for content-type breakdowns), min_impressions (default 10: filters out low-data experiments), and win_threshold (default 0.0: the minimum CTR lift required to count as a win).

How to Read the Report

The report goes from test results to a summary of all tests:

Experiment Verdicts. A table showing every experiment with its verdict: Win, Loss, or Inconclusive. Each row shows the control CTR, treatment CTR, absolute and relative lift, confidence interval, and p-value. Color-coded so wins are immediately visible. This is where you quickly see which changes worked.

Batch Comparison. If experiments were run in multiple batches (different time periods), this section compares batch-level performance. You can see if your experimentation quality is improving over time: is your win rate going up? Are your effect sizes getting larger as you learn from past results?

Effect Size Distribution. A histogram or density plot showing the distribution of CTR changes across all experiments. This tells you the typical magnitude of your experiments. If most experiments produce tiny effects, you might need bolder changes. If the distribution has a long right tail, your winning experiments are producing outsized gains.

Click Projection. Extrapolates the cumulative click impact of your winning experiments over time. If five experiments each gained 50 additional clicks per month, the annualized impact is 3,000 additional clicks. This section turns statistical results into business metrics that stakeholders understand.

Data Sufficiency. Flags experiments that may not have enough data for reliable conclusions. Low-impression experiments produce noisy CTR estimates. This section tells you which verdicts to trust and which need more data collection time.

TL;DR. Portfolio-level executive summary: win rate, average effect size, total projected click impact, and top action items.

When to Use Something Else

If you want to analyze a single SEO title experiment in depth (with forest plots, position-adjusted CTR comparisons, and detailed statistical tests) use the Title A/B Test module. It is designed for deep analysis of one experiment at a time, while Portfolio Analysis is designed for batch evaluation of many experiments.

If you are not running formal experiments but want to find SEO opportunities in your existing GSC data, use the GSC Quick Wins module. It identifies pages ranking 4-20 with high impressions but low CTR: natural candidates for your next batch of experiments.

If you want to detect organic ranking changes over time (not controlled experiments, but natural fluctuations), use the Ranking Changes module. It compares two time periods to find which pages gained or lost positions.