How MapleStats works with Statistics Canada

Many services, one way in

StatCan's data sits behind several services. The Web Data Service (WDS) serves tables and time series as JSON. An SDMX REST API serves the same tables as SDMX-ML. Reference Data as a Service (RDaaS) holds classifications such as NAICS. The 2021 Census Profile has its own SDMX API on another host, the 2016 profile has a separate JSON API, and the profiles from 2001 to 2016 exist as bulk downloads. Public use microdata files (PUMFs) are ZIP archives. Smaller services cover The Daily and its release calendar, the Data catalogue, the survey directory, geography, merchandise trade by commodity, the open databases and the daily bulk-update files.

Each service has its own identifiers, its own error behaviour and its own format for request bodies. An agent should not have to learn all of that before it can answer a question about prices. MapleStats wraps these services in 84 tools, and the agent does not see them as a list. The server shows a client three tools: plan_query, search_tools and call_tool. The agent finds each StatCan tool with a plain-language query to search_tools and runs it through call_tool. The first request of a StatCan question is therefore not a request to StatCan:

{
  "query": "search StatCan tables"
}

[
  "statcan_census_tables_search",
  "wds_search_cubes",
  "statcan_census_tables_get_downloads",
  "statcan_reference_search_data",
  "ab_economic_list_tables"
]

The ranking comes from the server's BM25 index over tool names and docstrings, the same index the search on this site uses. wds_search_cubes is among the results, and a table search starts there.

From a question to a table

Every StatCan table has a product ID, the PID. The PID is the table number without its dashes: table 18-10-0004-01 is PID 18100004, because the last two digits, a view of the table, are optional. The server's docs://statcan/addressing resource breaks down the full ten digits: two for the subject, two for the product type, four for the sequence and two for the view.

wds_search_cubes turns words into a PID with a simple method. It downloads getAllCubesListLite, the list of every table in WDS, caches it for an hour and looks for the query as a substring of each English and French title. That is how it finds the consumer price index table.

Substring matching has one clear limitation. "consumer price index" finds the CPI tables because their titles contain those words. A question worded differently from StatCan's titles can miss them, and the agent then has to search again in StatCan's terms. The provenance reports how much was searched. Each result carries cansim_id, the table's number in the old CANSIM system, and release_time, the table's latest release.

Coordinates and vectors

A StatCan table is a cube. Each dimension has a tree of members, and one member from each dimension identifies one series. wds_get_cube_metadata returns the dimensions. The CPI table has two: Geography, with 30 members, and Products and product groups, with 359.

Member IDs repeat across dimensions: in both, member 2 is the root of the tree, the member with no parent (Canada, and All-items). Not every combination exists. 30 geographies times 359 products would make 10,770 series, but the table has 2,139. A coordinate therefore has to name a series StatCan publishes.

A coordinate lists one member ID per dimension, in dimension order, separated by dots. WDS always expects exactly ten positions, with zeros for the dimensions a table does not have. The client adds the padding, so a request for 2.2 sends 2.2.0.0.0.0.0.0.0.0. The response is the vector: a stable ID for one series, the "V" number carried over from CANSIM.

With the vector, the data takes one more call. This is the request for the latest 84 months of v41690973:

{
  "name": "wds_get_data_from_vectors",
  "arguments": {"vector_ids": [41690973], "latest_n": 84}
}

[
  {
    "product_id": 18100004,
    "coordinate": "2.2.0.0.0.0.0.0.0.0",
    "vector_id": 41690973,
    "observations": [
      // 83 earlier months
      {
        "ref_period": "2026-08-01",
        "value": 169.8,
        "decimals": 1,
        "scalar_factor_code": 0,
        "symbol_code": 0,
        "status_code": 0,
        "security_level_code": 0,
        "release_time": "2026-09-14T08:30:00Z"
      }
    ],
    "provenance": {
      "url": "https://www150.statcan.gc.ca/t1/wds/rest/getDataFromVectorsAndLatestNPeriods"
      // + 9 more fields
    }
  }
]

Each observation comes with the codes needed to read it:

scalar_factor_code

The power of ten the value is expressed in: 0 for units, 3 for thousands, 6 for millions. For the CPI it is 0.

decimals

The number of decimals StatCan publishes. The value already carries StatCan's rounding.

symbol_code, status_code

The symbols and status flags StatCan attaches to a value, as codes. wds_get_code_sets decodes them, along with scalar factors, frequencies and units of measure.

release_time

The release timestamp WDS gives the data point. It is not always the first publication: August 2026 carries 14 September 2026, but September 2019, the oldest month in this recording, carries 15 September 2021.

One rule matters more to me than the others: the server never applies the scalar factor. value is exactly what WDS sent. The code states this twice, in the schema's docstring ("deliberately NOT applied ... WDS never auto-applies it either") and first on the list in docs://statcan/gotchas. A value in thousands stays in thousands, with its code beside it. Scaling takes one line: apply_scalar_factor(value, code) multiplies by ten to the power of the code. I leave that line to whoever uses the number, where it stays visible. The tool returns what WDS returns, digit for digit.

Slicing tables with SDMX

WDS works in series: give it vectors or coordinates and it returns their observations. Some questions cover a whole slice of a large table, such as every detailed geography or every occupation. Going series by series then means finding every coordinate first, and downloading the full table means taking far more data than the question needs. StatCan's SDMX API does the slicing on the server. A key names the members you want, dimension by dimension, and a blank position is a wildcard.

The key is the coordinate without its padding: one position per non-time dimension. sdmx_get_vector_data builds the key from a vector. It looks up the vector's coordinate through WDS, reads the table's SDMX structure to count its dimensions and cuts the coordinate to that length. v41690973 becomes the key 2.2.

The client handles three quirks of this API.

It answers in XML

StatCan's SDMX endpoint returns SDMX-ML for both data and structure, even when a request asks for JSON with format=jsondata or an Accept header. The constants file records this as confirmed live, against benchmark documentation that assumed SDMX-JSON. The client therefore parses the XML itself, with defusedxml rather than the standard library parser.

Wildcards sample big dimensions

If a dimension with more than about 30 codes is left blank, the response holds a sparse, unpredictable sample of its codes, not all of them. sdmx_get_key_for_dimension builds the complete version instead. It reads the dimension's codelist from the structure, keeps the leaf codes (those that are no other code's parent) and joins them with + into an explicit OR key, which goes into the key at that position.

One combination is refused

lastNObservations together with startPeriod or endPeriod gets HTTP 406. The client rejects that combination before sending anything. If a 406 comes back anyway, the error names the usual cause instead of passing on the bare status.

Microdata, weights and replicates

Tables are aggregates. A public use microdata file holds the records themselves, one row per respondent, with weights that scale the sample up to the population. StatCan ships PUMFs as ZIPs, and the codebooks inside come in several formats: CSV codebooks, Stata .dct and .do files, SPSS label files and SAS files.

The table APIs do not cover PUMFs. The only place to find them is StatCan's Data catalogue, which statcan_reference_search_data searches. Once the agent has a file, statcan_pumf_get_codebook reads its codebook from inside the ZIP with HTTP range requests, so the variables and weights are known before the data file is downloaded. statcan_pumf_tabulate then computes on the server with DuckDB. Its first call downloads the file into a local cache, and later calls reuse it.

This is the call behind the bachelor's degree chart on the demos page: the 2021 Census individuals file, adults aged 25 to 64 (age groups 9 to 16), and the share holding each highest credential in each province, with the codes for "not available" and "not applicable" filtered out.

{
  "name": "statcan_pumf_tabulate",
  "arguments": {
    "url": "https://www150.statcan.gc.ca/n1/pub/98m0001x/2023001/cen21_ind_98m0001x_part_rec21.zip",
    "rows": [
      "PR",
      "HDGREE"
    ],
    "statistic": "share",
    "filters": {
      "AGEGRP": ["9", "10", "11", "12", "13", "14", "15", "16"],
      "HDGREE": ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13"]
    }
  }
}

{
  "data_file": "data_donnees_2021_ind_v2.csv",
  "statistic": "share",
  "weight": "WEIGHT",
  "unweighted_n": 525504,
  "weighted_total": 19463104.06541252,
  "cells": [
    {
      "groups": [
        {"variable": "PR", "code": "10", "label": "newfoundland and labrador"},
        {"variable": "HDGREE", "code": "9", "label": "bachelor's degree"}
      ],
      "estimate": 13.305555555555555,
      "standard_error": 0.42801216439502804,
      "cv": 0.032167928848060565,
      "unweighted_n": 958,
      "low_count": false
    }
    // 142 more cells
  ]
  // + url, value_variable, filters, truncated, variance_method, notes, provenance
}

Two things determine whether a table like this one is correct. The first is the weight. The tool defaults to the codebook's main weight, WEIGHT here. It checks labels and field widths as well as names, because names can mislead: in the CSWC, WTQ_05 is a question about working at night, and in the CCHS, DOHWT is a height-and-weight inclusion flag. Real weights are wide numeric fields, so a known width under four columns rules a variable out. The second is the sample behind each cell. Every cell carries its unweighted count, and by default the tool flags cells with fewer than 30 respondents as low_count.

For standard errors, the Census file carries 16 replicate weights, WT1 to WT16. The tool computes the estimate once with the main weight and once with each replicate. It then sums the squared deviations of the 16 replicate estimates from their mean, divides by 35 and takes the square root. The divisor of 35 comes from the user guide: it is a Fay adjustment over 16 groups, and the comment above it in tabulate.py shows the arithmetic, (240/35) × (1/240). StatCan notes that this method overestimates the error for small estimates.

I copy each method from its survey's user guide and cite the section. A file gets a method only after I have read its guide, "never by analogy with another survey", as the comment above the list puts it. 3 files have one so far:

For any other PUMF, the result gives no standard error, says why and names the replicate weights or bootstrap file it found. PUMF bootstrap weights are perturbed for confidentiality, so their standard errors are comparable to StatCan's official ones but not identical. Two demos use the 2021 Census file this way: bachelor's degrees by province and low income across immigrant generations.

Quirks from live calls

I found most of these by calling the live services, not by reading documentation. Next to each fix, the code records how I confirmed the problem, so that a later edit does not quietly undo it.

From a call to a script

An agent's answer is only as good as the check someone can run without the agent. reproduce_code takes a tool name and its arguments and writes an R, Python, Stata or Julia script (or an Excel Power Query) that fetches the same data directly from StatCan. What the script does depends on the call:

For the CPI call above, the R script fetches the data with one cansim call, and the Python script repeats the WDS request, http2=True included. Both follow the scalar rule: they keep value as WDS sent it and put the scaled number in a separate column, val_norm from cansim in R and value_normalized in Python. From the recorded scripts:

with httpx.Client(
    http2=True,
    follow_redirects=True,
    timeout=300,
    headers={"User-Agent": "Mozilla/5.0 (compatible; research script)"},
) as client:
    response = client.post('https://www150.statcan.gc.ca/t1/wds/rest/getDataFromVectorsAndLatestNPeriods', json=[{'vectorId': 41690973, 'latestN': 84}])
response.raise_for_status()
raw_path.write_bytes(response.content)

# …

# scalarFactorCode is the value's power of ten; refPer is the reference date.

data = data.with_columns(
    (pl.col("value") * 10 ** pl.col("scalarFactorCode")).alias("value_normalized"),
    pl.col("refPer").str.to_date(strict=False).alias("ref_date"),
)

data <- get_cansim_vector(c("v41690973"))

# …

# cansim adds val_norm (the value times its scalar factor) and a Date column.

data <- data |>
  filter(!is.na(val_norm))

The other services

Classifications. RDaaS holds NAICS, the Standard Geographical Classification and the rest: structure, category trees, index terms, exclusions, and concordances that map codes from one version to the next. It has one gap. Asked for the category tree of the current NAICS, 2022.1.0, it answers with an empty body, while every older version tried returns the full tree. The tool says so and points to the 2017-to-2022 concordance, whose target codes are the current NAICS codes. Index terms switch to French only through an Accept-Language header, which the client sends.

The census. The 2021 Census Profile is an SDMX API on its own host. A query picks a geography, from a province down to a dissemination area, and one of 2,631 characteristics, then fetches the values. This API also picks its language from an Accept-Language header rather than a parameter. The 2016 profile has a separate JSON API, and the 2001 to 2016 profiles are CSV or TAB bulk downloads, which the archive tools resolve to direct links. Census geography comes from an ArcGIS REST service that returns boundaries and DGUIDs. It answers every error with HTTP 200 and an error object inside.

Releases. The Daily comes from its Atom feeds, the last 100 days by subject, and from the JSON file behind the release calendar, which goes back to 14 March 2012. wds_get_changed_cube_list lists the tables that changed on a date, and statcan_delta_get_file_link finds the bulk-update ZIP for one business day; statcan_delta_read_table reads one table's new data points from that ZIP without downloading all of it. statcan_daily_get_release_calendar gives the dates of upcoming releases of key indicators.

Trade and other SDMX spaces. cimt_get_trade reads the database behind StatCan's merchandise trade application: monthly exports and imports by HS commodity, partner country, US state and province, a level of detail WDS does not carry. sdmx_space_get_data reaches two more SDMX spaces, one for the Canadian Centre for Energy Information and one for indicators StatCan shares with other departments.

The full list of families, with each one's prefix and number of tools, counted from the registry when this page was built:

Where to go next

• All 84 Statistics Canada toolsEvery tool's parameters, keywords and an example call.
• The demosThe Census calls on this page as charts, with the calls behind each one.
• Connect MapleStats to your agentOne address, no account or API key.