How to Use Sales Anomaly Detection in Shopify: Step-by-Step Tutorial

Introduction to Sales Anomaly Detection

Every Shopify store owner has asked the question: "Are there any unusual patterns in our sales data?" Whether it is an unexpected revenue spike, a mysterious drop in orders, or a day you want to explain, anomaly detection gives you a short list of days that do not fit the usual pattern, so you can look at those first.

To find the rows in your own data that do not fit the usual pattern, try the free anomaly detection tool.

The method in this tutorial is simple and checkable. For each day it asks how far that day's revenue sits from what is normal for that weekday, measured in units of the typical day-to-day spread. A day far outside that spread is flagged. It is a screen for days worth a look, not a verdict: a flagged day can be a campaign, an outage, a data problem, or just chance, and the tutorial measures how often chance alone produces a flag.

The figures in every output block come from a synthetic data set generated for this tutorial (120 days of orders with three events planted on purpose), not from a real store, so you can compare what the method finds with what we know is there. Your numbers will differ.

What You'll Accomplish

In this tutorial, you will:

• Export your Shopify orders and total them by day
• Build a weekday baseline and a robust z-score for each day
• Choose a threshold and read the days it flags
• Measure how many false alarms that threshold produces on clean data
• Test the setup by planting a known spike
• Investigate flagged days and re-run the analysis on a fresh export

Difficulty Level: Intermediate

Prerequisites and Data Requirements

What You Need Before Starting

1. Shopify Store Access: Permission to export orders from your store's admin
2. Order History: Enough days that each weekday appears many times. This tutorial uses 120 days, which gives 17 or 18 of each weekday; with only four weeks each weekday baseline rests on four values and moves a lot from one export to the next
3. Python with pandas and NumPy, or a spreadsheet if you rebuild the same steps there
4. Basic Statistics Knowledge: Mean, median and spread (explained as we go)

Data Quality Checklist

• No long gaps in order history (a few days without orders is normal and counts as zero revenue)
• Timestamps on every order
• One currency, or totals already converted to one
• A payment status column, so refunded and pending orders can be left out

Poor data quality produces both false flags and missed events, so check the export before trusting any flag.

Step 1: Export Your Shopify Orders

1. In your Shopify admin, open the Orders page
2. Use its Export option (the menu labels change between admin versions, so follow Shopify's current help page for exporting orders)
3. Export all orders for the period you want to analyze, as CSV

Shopify's order CSV has one row per line item, with the order-level fields (date, status, total) filled on the first row of each order only. Check the header row of your own file: the code below assumes the columns Created at, Financial Status and Total.

Expected Output

Name,Created at,Financial Status,Total,Lineitem quantity
#1001,2025-09-01 00:07:41 +0000,paid,38.12,1
#1001,,,,1
#1002,2025-09-01 00:52:19 +0000,paid,71.40,1

These rows are illustrative. An order with two items shows a second row with blank order-level fields, which is why the next step keeps only rows that have a date.

Step 2: Total Paid Orders by Day

This code keeps the first row of each order, drops orders that are not paid, converts timestamps to UTC dates, and adds up revenue per day. Days with no orders are filled in as zero, because a day with no sales is exactly what a drop looks like. Dates here are UTC days; if your store's time zone differs, convert before grouping so late-evening orders land on the right day.

import numpy as np
import pandas as pd

df = pd.read_csv("orders_export.csv")
# Order-level fields (date, status, total) sit on the first row of each order
orders = df.dropna(subset=["Created at"]).copy()
orders["day"] = pd.to_datetime(orders["Created at"], utc=True).dt.tz_localize(None).dt.normalize()
orders = orders[orders["Financial Status"] == "paid"]

daily = orders.groupby("day")["Total"].agg(revenue="sum", orders="size")
daily = daily.reindex(pd.date_range(daily.index.min(), daily.index.max()), fill_value=0)
print(f"Paid orders: {int(daily['orders'].sum()):,}")
print(f"Days: {len(daily)} ({daily.index.min().date()} to {daily.index.max().date()})")
print(f"Revenue: ${daily['revenue'].sum():,.2f}")

Expected Output

Paid orders: 2,658
Days: 120 (2025-09-01 to 2025-12-29)
Revenue: $174,345.58

Step 3: Set a Weekday Baseline and a Threshold

The Baseline

Sales differ by weekday, so each day is compared with the median revenue for its own weekday, not with one overall average. The difference between actual and baseline is the residual.

The Score

A standard deviation measures how spread out values are. A z-score is different: it describes one day, as the number of spread units between that day and the baseline. Here the spread is the median absolute deviation (MAD) of the residuals, multiplied by 1.4826 so that it matches a standard deviation when the residuals are normally distributed. The median-based version is used because a few extreme days would inflate an ordinary standard deviation and hide themselves. This is a robust z-score.

The Threshold

• 3: flags only large departures; fewer false alarms, but small real events are missed
• 2: flags many more days; use it only if you are willing to look at several days that turn out to be ordinary (Step 5 measures how many)

Daily totals suit stores with enough orders per day that one day's revenue is not dominated by a single order. For a store with a handful of orders a day, weekly totals are steadier, but then you have far fewer points: a 90-day history is about 13 weeks, so the weekday baseline no longer applies and each week would be compared with the overall median. Monthly totals leave too few points for this method.

Step 4: Read the Flagged Days

The code scores every day and lists those beyond the threshold. deviation_pct is actual revenue against the weekday baseline.

def score(revenue):
    """Robust z-score of each day against the median for its weekday."""
    weekday = revenue.index.dayofweek
    expected = revenue.groupby(weekday).transform("median")
    resid = revenue - expected
    scale = 1.4826 * (resid - resid.median()).abs().median()  # MAD scaled to match a standard deviation
    return expected, resid / scale

daily["expected"], daily["z"] = score(daily["revenue"])
THRESHOLD = 3.0
flagged = daily[daily["z"].abs() > THRESHOLD]
print(f"Days flagged at |z| > {THRESHOLD}: {len(flagged)} of {len(daily)}")
out = flagged.assign(deviation_pct=(flagged["revenue"] / flagged["expected"] - 1) * 100)
print(out[["revenue", "expected", "deviation_pct", "z"]].round(1).to_string())

Expected Output

Days flagged at |z| > 3.0: 3 of 120
            revenue  expected  deviation_pct    z
2025-10-11    108.8    1683.4          -93.5 -4.5
2025-11-15   4282.6    1683.4          154.4  7.4
2025-12-14   3233.3    1690.3           91.3  4.4

Three days are flagged: a Saturday with almost no revenue (the outage planted in this data), a Saturday at 2.5 times its baseline (the flash sale), and a Sunday at 1.9 times (the smaller spike). In the data set we generated, all three flagged days are planted events. In your own export you will not know which flags are real, which is why Step 5 measures the false-alarm rate.

Possible Causes to Check

Revenue spikes

• A promotion or ad campaign, or a mention on social media or in the press
• A pricing error or an unintended discount
• Suspicious orders: a sudden run of orders from new customers deserves a look

Revenue drops

• Site downtime, checkout or payment-gateway problems
• Out-of-stock products or a fulfillment delay
• A paused campaign or lower ad spend
• An export problem: check that the day's orders are in the file

Step 5: Measure Your False-Alarm Rate

A threshold in z-score units does not tell you how many false flags to expect, because real revenue is not exactly normal. This code builds 300 clean 120-day series that use the same weekday order counts and the same order values as your data but contain no planted events, scores them with the same function, and counts the flags. Every flag on a clean series is a false alarm by construction. (It is a simulation of your store with its unusual days removed, so it is only as good as the assumption that orders arrive independently.)

rng = np.random.default_rng(7)
order_values = orders["Total"].to_numpy()
mean_orders = daily.groupby(daily.index.dayofweek)["orders"].median()  # median, so the planted events do not inflate the rate

def clean_series(rng):
    """120 days with the same weekday pattern and order values, and NO planted events."""
    idx = pd.date_range("2025-01-06", periods=120)
    counts = [rng.poisson(mean_orders[d.dayofweek]) for d in idx]
    return pd.Series([rng.choice(order_values, c).sum() for c in counts], index=idx)

for thr in (2.0, 3.0):
    flags = [(score(clean_series(rng))[1].abs() > thr).sum() for _ in range(300)]
    print(f"|z| > {thr}: {np.mean(flags):.1f} false alarms per 120 clean days ({np.mean(flags) / 120 * 100:.1f}% of days)")

Expected Output

|z| > 2.0: 8.1 false alarms per 120 clean days (6.7% of days)
|z| > 3.0: 1.2 false alarms per 120 clean days (1.0% of days)

With the cutoff at 3, about one clean day in a hundred is still flagged, about 1.2 false alarms per 120 days. That is more than the 0.27% (about 0.3 days in 120) a normal distribution would give, because order revenue has a heavier tail than a normal curve. With the cutoff at 2, about 7% of clean days are flagged, roughly 8 false alarms in 120 days, so a list of flagged days at that setting will mostly be ordinary days. Pick the cutoff by how many false alarms you are willing to investigate, and expect them.

Step 6: Investigate and Re-Run

Test the Setup With a Planted Spike

Before trusting the flags on your own data, plant a known event and check that it is found. This multiplies one ordinary day's revenue by three and re-scores:

test = daily["revenue"].copy()
day = pd.Timestamp("2025-10-08")
test[day] = test[day] * 3
_, z_test = score(test)
print(f"Planted day {day.date()}: z = {z_test[day]:.1f}, flagged = {abs(z_test[day]) > THRESHOLD}")

Expected Output

Planted day 2025-10-08: z = 5.9, flagged = True

A planted tripling is found with room to spare. A smaller change is not: in this data set, doubling the same day's revenue gives a z-score of 2.65, still under 3, because a single day has to move a long way against this much day-to-day noise. The method finds large departures, not subtle ones.

Investigation Checklist

For each flagged day:

1. Data: confirm the day's orders in the Shopify admin match the export; check time zone and refunds
2. Site: look at your traffic, downtime and checkout records for that day
3. Business context: check your campaign calendar, stock levels and known events
4. Record: note the cause you found, or that you found none

Lining flagged days up with your campaign calendar shows association, not effect: a spike on a campaign day does not tell you what sales would have been without the campaign. Measuring that needs a comparison group or a planned test.

Re-Running It

The same code runs on any fresh export, so a weekly or monthly re-export and re-run is a routine you control. When your business changes (a new product line, a permanent price change), the old weekday medians stop describing normal, so start the baseline from the date of the change.

Check That It Works

1. The planted spike is found (Step 6)
2. The number of flagged days is close to the false-alarm count from Step 5 or higher. If you flag far more days than the simulation predicts for clean data, your series has real structure the baseline misses, such as a trend or promotions; if you flag none, that is also a reasonable result for a quiet period
3. You can name a cause for most flagged days. The share you can explain is a number to track for your own store; this tutorial does not claim a benchmark for it

Run It on Your Own Shopify Export

Export your orders from Shopify as a CSV and upload it at MCP Analytics. The analysis runs robust z-score anomaly detection on your daily revenue and shows the result with the method used. You choose the columns; nothing connects to your Shopify account.

Try it on your data

Next Steps with Shopify Analytics

Expand Your Analysis

1. Product-level series: apply the same function to revenue or units for one product or category
2. Other measures: score orders per day, average order value or refund count the same way
3. Regional series: score revenue by shipping country or region

Related Resources

• A/B Testing Statistical Significance - for testing the cause of a change you found

Troubleshooting Common Issues

Issue 1: Too Many Flags

Symptom: Most flagged days turn out to be ordinary.

• Raise the cutoff from 2 to 3 (Step 5 shows the false-alarm cost of each)
• Check whether a promotion calendar or a trend is moving the baseline, and start the baseline after a permanent change
• Use weekly totals if daily orders are few

Issue 2: A Known Event Was Not Flagged

• The event may be too small relative to normal day-to-day spread; lower the cutoff and accept more false alarms
• A recurring event, such as a sale every Friday, becomes part of the weekday median and stops looking unusual, which is the correct behavior for an expected event
• Check that the day's orders are in the export and the status filter did not drop them

Issue 3: Results Change Between Runs

• Scoring is deterministic for the same file; a different result means a different export (new orders, refunds changing status) or a different date range
• The false-alarm simulation uses random numbers; fix the seed (default_rng(7)) to repeat it exactly

Issue 4: The CSV Will Not Load

• Check the column names in your header row against the ones the code uses
• Remove summary rows or footer text from the file
• Make sure Total holds plain numbers, not text with currency symbols

Issue 5: No Days Flagged

• With a cutoff of 3 and a quiet period, zero flags is a normal result
• With too little history the baseline is noisy; add more weeks